A CPE-based XR data processing method, device, equipment and medium

By judging and performing different operations on XR system components, and using CPE devices for multi-terminal data transmission, video multicast and rendering calculations, the use limitations of XR streaming applications are solved, reducing the hardware cost of terminal devices and improving the application experience.

CN117978995BActive Publication Date: 2024-11-19GUANGZHOU TOZED KANGWEI INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410038921.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2023-08-01
Filing Date
2024-01-10
Publication Date
2024-11-19
Estimated Expiration
2044-01-10

AI Technical Summary

Technical Problem

The existing XR streaming technology has high hardware costs for terminal devices and cannot support terminal devices without high hardware configurations, resulting in large usage restrictions.

Method used

By judging the components of the XR system, if the components are cloud servers, CPE devices and terminal devices, multi-end data transmission or video multicast operations are performed; if the components are cloud servers and CPE devices, multi-device data transmission operations are performed; if the components are cloud servers and CPE devices, multi-device data transmission operations are performed; if the components are cloud servers, multi-device data transmission operations are performed; For CPE devices and terminal devices, rendering calculation operations are performed, and CPE devices are used to break through the usage limitations of streaming applications and reduce the hardware cost of terminal devices.

Benefits of technology

It realizes the simultaneous operation and data interaction of multiple XR devices in the same area, supports synchronous video playback, reduces the hardware cost of terminal devices, and provides a more stable and low-latency XR application experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117978995B_ABST
    Figure CN117978995B_ABST
Patent Text Reader

Abstract

The present invention discloses a CPE-based XR data processing method, device, equipment and medium. By judging the composition of XR system components, if the components are a first cloud server, a first CPE device and a first XR terminal device, the first CPE device performs a multi-segment data transmission operation or a video multicast operation; if the components are a second cloud server and a second CPE device, the second CPE device performs a multi-device data transmission operation; if the components are a third CPE device and a third XR terminal device, the third XR terminal device performs a rendering calculation operation. The present invention identifies the composition of XR system components and analyzes the application scenarios of the XR system, thereby performing XR data transmission for clients and servers with different settings, breaking through the usage restrictions of XR streaming applications with CPE devices and reducing the hardware cost of terminal devices.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of virtual reality, and in particular to a CPE-based XR data processing method, device, equipment and medium. Background Art

[0002] XR (Extended Reality) streaming technology is a technology that combines XR technologies such as virtual reality, augmented reality, and mixed reality with cloud computing, network transmission and other technologies. XR applications require high hardware computing power, while wireless terminals usually have limited computing power and cannot support the operation of high-performance XR applications. The cloud has sufficient computing power and storage resources, so the cloud implements the computing tasks of high-performance XR applications and streams the rendered video stream data to the terminal, so that high-performance XR applications can be experienced through low-cost, low-computing terminal devices. At present, the mainstream commercial XR rendering computing offload is implemented by the cloud, and the server streaming application is deployed in the cloud, and the client streaming application is deployed in the terminal. The server performs rendering calculations on the XR application based on the posture, motion, control, environment and other information uploaded by the client, and finally transmits the rendered image data to the client, thereby realizing the streaming of XR applications.

[0003] The streaming of high-performance XR applications requires high communication bandwidth, so the terminal devices that implement high-performance XR application streaming should have strong communication capabilities with the cloud, which increases the terminal hardware cost. In addition, some terminal devices without high hardware configuration cannot run XR streaming applications, which increases the usage restrictions of XR streaming applications.

[0004] Therefore, there is an urgent need for an XR data processing strategy to solve the problem of large usage restrictions on XR streaming applications. Summary of the invention

[0005] Embodiments of the present invention provide a CPE-based XR data processing method, apparatus, device, and medium to solve the problem of large usage restrictions on XR streaming applications.

[0006] In order to solve the above problem, an embodiment of the present invention provides a CPE-based XR data processing method, including:

[0007] Make judgments about the components of the XR system;

[0008] If the components are a first cloud server, a first CPE device and a first XR terminal device, the first CPE device performs a multi-terminal data transmission operation or a video multicast operation; wherein the first cloud server is provided with a server, and the first XR terminal device is provided with a client;

[0009] If the component is a second cloud server and a second CPE device, the second CPE device performs a multi-device data transmission operation; wherein the second cloud server is provided with a server, and the second CPE device is provided with a client;

[0010] If the components are a third CPE device and a third XR terminal device, the third XR terminal device performs rendering calculation operations; wherein the third CPE device is provided with a server, and the third XR terminal device is provided with a client.

[0011] As an improvement of the above solution, the first CPE device performs a multi-terminal data transmission operation or a video multicast operation, including:

[0012] Receiving video data transmitted by the first cloud server;

[0013] Making a judgment on the video data;

[0014] If the type of the video data is a rendered video, a multi-end data transmission operation is performed;

[0015] If the type of the video data is cached video, a video multicast operation is performed.

[0016] As an improvement of the above solution, the multi-terminal data transmission operation includes:

[0017] The first CPE device generates a communication channel so that the first cloud server is connected to several first XR terminal devices respectively through the communication channel; wherein, the server corresponding to the first cloud server renders the first operation data transmitted by the client corresponding to each first XR terminal device, generates first rendered video data corresponding to each first operation data, and transmits each first rendered video data to the corresponding first XR terminal device; the client corresponding to each first XR terminal device displays according to the first rendered video data.

[0018] As an improvement of the above solution, the video multicast operation includes:

[0019] The first CPE device receives cached video data transmitted by the first cloud server;

[0020] The first CPE device transmits the cached video data to a plurality of the first XR terminal devices in a multicast manner, so that the first XR terminal devices display according to the cached video data.

[0021] As an improvement of the above solution, the multi-device data transmission operation includes:

[0022] The client corresponding to the second CPE device receives the second operation data transmitted by the second XR terminal device;

[0023] The client corresponding to the second CPE device transmits the plurality of second operation data to the second cloud server, so that the second cloud server renders each of the second operation data to obtain second rendered video data corresponding to each of the second operation data;

[0024] The client corresponding to the second CPE device receives a plurality of the second rendered video data, and transmits each second rendered video data to the corresponding second XR terminal device, so that the second XR terminal device displays according to the second rendered video data.

[0025] As an improvement of the above solution, the rendering calculation operation includes:

[0026] The server corresponding to the third CPE device receives the third operation data transmitted by the client corresponding to each of the third XR terminal devices;

[0027] The server corresponding to the third CPE device renders each of the third operation data to obtain third rendered video data corresponding to each of the third operation data;

[0028] The server corresponding to the third CPE device transmits each of the third rendered video data to the corresponding third XR terminal device, so that the third XR terminal device displays according to the third rendered video data.

[0029] Accordingly, an embodiment of the present invention further provides a CPE-based XR data processing device, comprising: a data determination module, a first processing module, a second processing module and a third processing module;

[0030] The data judgment module is used to judge the components of the XR system;

[0031] The first processing module is configured to, if the components are a first cloud server, a first CPE device and a first XR terminal device, the first CPE device performing a multi-terminal data transmission operation or a video multicast operation; wherein the first cloud server is provided with a server, and the first XR terminal device is provided with a client;

[0032] The second processing module is used for, if the component is a second cloud server and a second CPE device, the second CPE device performs a multi-device data transmission operation; wherein the second cloud server is provided with a server, and the second CPE device is provided with a client;

[0033] The third processing module is used for, if the components are a third CPE device and a third XR terminal device, the third XR terminal device performs rendering calculation operations; wherein, the third CPE device is provided with a server, and the third XR terminal device is provided with a client.

[0034] As an improvement of the above solution, the first CPE device performs a multi-terminal data transmission operation or a video multicast operation, including:

[0035] Receiving video data transmitted by the first cloud server;

[0036] Making a judgment on the video data;

[0037] If the type of the video data is a rendered video, a multi-end data transmission operation is performed;

[0038] If the type of the video data is cached video, a video multicast operation is performed.

[0039] Correspondingly, an embodiment of the present invention further provides a computer terminal device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements a CPE-based XR data processing method as described in the present invention.

[0040] Correspondingly, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a CPE-based XR data processing method as described in the present invention.

[0041] As can be seen from the above, the present invention has the following beneficial effects:

[0042] The present invention provides an XR data processing method based on CPE, by judging the composition of XR system components, if the components are a first cloud server, a first CPE device and a first XR terminal device, the first CPE device performs a multi-terminal data transmission operation or a video multicast operation; if the components are a second cloud server and a second CPE device, the second CPE device performs a multi-device data transmission operation; if the components are a third CPE device and a third XR terminal device, the third XR terminal device performs a rendering calculation operation. The present invention identifies the composition of XR system components and analyzes the application scenarios of the XR system, thereby performing XR data transmission for clients and servers with different settings, breaking through the usage restrictions of XR streaming applications with CPE devices and reducing the hardware cost of terminal devices.

[0043] Preferably, the present invention performs multi-terminal data transmission operations through CPE devices, realizes data transmission between a cloud server equipped with a server and multiple XR terminal devices equipped with clients, and supports multiple XR devices in the same area to simultaneously run XR applications and interact with data; performs video multicast operations through CPE devices, forwards cached videos stored in the cloud server to multiple XR terminal devices, so that the videos of the terminal devices are played synchronously; performs multi-device data transmission operations through CPE devices, and transmits the rendered video data of the server to multiple XR terminal devices based on the connection between one server and one client, thereby realizing data sharing of client-free XR terminal devices; performs data rendering operations through CPE devices, without the need to connect to a cloud server, and can provide a more stable XR application experience with lower latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flowchart of a CPE-based XR data processing method provided by an embodiment of the present invention;

[0045] Figure 2 is a schematic structural diagram of a CPE-based XR data processing device provided by an embodiment of the present invention;

[0046] Figure 3 It is a schematic diagram of the structure of a terminal device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Embodiment 1

[0049] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a CPE-based XR data processing method provided by an embodiment of the present invention. Figure 1 As shown, this embodiment includes steps 101 to 104, and each step is specifically as follows:

[0050] Step 101: Determine the components of the XR system.

[0051] In a specific embodiment, a cloud server, a CPE device, and an XR terminal device.

[0052] The cloud server is a server device with sufficient computing power, storage, and bandwidth resources. The cloud server is responsible for implementing the storage and rendering computing functions of XR application data, as well as the streaming video distribution function. For example, it stores XR videos, models, applications and other related data; as the server of the XR streaming solution, it processes the XR rendering computing tasks deployed in the cloud and pushes the rendering results to the client of the XR streaming solution.

[0053] The CPE device of this embodiment adopts an enhanced CPE. The enhanced CPE adds a computing control unit device on the basis of the traditional CPE to provide wireless communication and computing control capabilities for the XR system. The CPE uses 4G / 5G uplink and WIFI downlink communication. The computing control unit of the enhanced CPE can be connected to the system through a network cable or other wired connection, thereby realizing fast data interaction with the CPE. The computing control unit can be designed as a variety of hardware platforms according to the needs of end users, such as chips that support x86, arm, and risc-v instruction set architectures, and support operating systems such as Windows, Liunx, and Andriod.

[0054] XR terminal devices are terminal devices that support the deployment of XR applications, such as mobile phones, tablets, AR / VR helmets, AR / VR glasses, etc. As the sending end, the XR terminal device is responsible for collecting the image data, motion posture, environmental status and other information of the user end, and uploading the relevant information to the enhanced CPE end or cloud server to provide reference information for the rendering calculation of the XR application; as the receiving end, the XR device is responsible for receiving the video stream data sent by the CPE, and processing and displaying the video data.

[0055] Step 102: If the components are a first cloud server, a first CPE device and a first XR terminal device, the first CPE device performs a multi-terminal data transmission operation or a video multicast operation; wherein the first cloud server is provided with a server, and the first XR terminal device is provided with a client.

[0056] In this embodiment, the first CPE device performs a multi-terminal data transmission operation or a video multicast operation, including:

[0057] Receiving video data transmitted by the first cloud server;

[0058] Making a judgment on the video data;

[0059] If the type of the video data is a rendered video, a multi-end data transmission operation is performed;

[0060] If the type of the video data is cached video, a video multicast operation is performed.

[0061] In this embodiment, the multi-terminal data transmission operation includes:

[0062] The first CPE device generates a communication channel so that the first cloud server is connected to several first XR terminal devices respectively through the communication channel; wherein, the server corresponding to the first cloud server renders the first operation data transmitted by the client corresponding to each first XR terminal device, generates first rendered video data corresponding to each first operation data, and transmits each first rendered video data to the corresponding first XR terminal device; the client corresponding to each first XR terminal device displays according to the first rendered video data.

[0063] In a specific embodiment, the enhanced CPE is connected to the cloud server via a wide area network and to the XR terminal device via a local area network, providing a communication channel for the cloud server and XR terminal device of the XR system. The XR terminal device collects local motion, posture and other information and uploads it to the XR server. The XR terminal device deploys the XR client application and uploads the terminal's motion, environment and other information to the cloud server. Receive the streaming video data sent from the server, and process and display it. The specific implementation steps are as follows:

[0064] The cloud server runs the XR server application, the XR terminal runs the XR client application, and the XR device accesses the WAN through the enhanced CPE;

[0065] The XR client accesses the XR server according to the network address and establishes a network connection between the server and the client;

[0066] The XR client uploads local motion, posture, environment and other information to the cloud server, and requests the server to send rendered video data;

[0067] The XR server application performs rendering calculations based on the relevant information uploaded by the XR client, and sends the rendered video data to the client;

[0068] The client receives the video data sent by the server and displays it.

[0069] Because in the current mainstream XR streaming solution, the server and the client are connected one-to-one, and the establishment of one-to-many connections cannot be supported. After the CPE device in this embodiment performs multi-terminal data transmission operations, if you want to support cloud streaming solutions for multiple XR applications in the same area, you need to deploy multiple XR streaming server applications with the server and establish multiple XR streaming connection channels. Deploying multiple XR server streaming applications on the server can be achieved by deploying virtualization technologies such as multiple virtual machines and multiple containers, or by deploying multiple hosts. Among them, CPE provides large bandwidth support for this solution. This solution supports multiple XR devices in the same area to run XR cloud streaming applications at the same time.

[0070] In this embodiment, the video multicast operation includes:

[0071] The first CPE device receives cached video data transmitted by the first cloud server;

[0072] The first CPE device transmits the cached video data to a plurality of the first XR terminal devices in a multicast manner, so that the first XR terminal devices display according to the cached video data.

[0073] In a specific embodiment, the XR video data is stored in the cloud server, and the enhanced CPE is connected to the cloud server via a wide area network and connected to the XR terminal device via a local area network. The XR video data is sent from the cloud to the enhanced CPE, and the enhanced CPE multicasts the cached XR video data to the XR terminal device connected to the local area network.

[0074] The specific implementation steps are as follows:

[0075] The XR terminal device sends a video data sending request;

[0076] The cloud server sends the XR video data to the enhanced CPE.

[0077] The enhanced CPE sends the XR video obtained from the cloud server to each XR device via multicast;

[0078] Each XR terminal device obtains the video data sent by the enhanced CPE and processes and displays it.

[0079] After the CPE device performs the video multicast operation, in application scenarios such as collective viewing of XR videos and XR live broadcasts, it only needs to obtain one video data from the cloud to the enhanced CPE and perform multicast. This solution can reduce the wide area network communication bandwidth requirements of XR applications and realize synchronous video playback.

[0080] Step 103: If the components are a second cloud server and a second CPE device, the second CPE device performs a multi-device data transmission operation; wherein the second cloud server is provided with a server, and the second CPE device is provided with a client.

[0081] In this embodiment, the multi-device data transmission operation includes:

[0082] The client corresponding to the second CPE device receives the second operation data transmitted by the second XR terminal device;

[0083] The client corresponding to the second CPE device transmits the plurality of second operation data to the second cloud server, so that the second cloud server renders each of the second operation data to obtain second rendered video data corresponding to each of the second operation data;

[0084] The client corresponding to the second CPE device receives a plurality of the second rendered video data, and transmits each second rendered video data to the corresponding second XR terminal device, so that the second XR terminal device displays according to the second rendered video data.

[0085] In a specific embodiment, an XR server application is deployed on a cloud server, and the XR application deployed on the cloud server performs rendering calculations based on the calculation reference information such as terminal motion and posture sent by the XR client. The rendered video data is sent to the XR client via streaming. The enhanced CPE deploys the XR client, connects to the cloud server via a wide area network, and connects to the XR terminal device via a local area network, providing a communication channel for the XR streaming system. The enhanced CPE receives local motion, posture and other information collected by the XR terminal device, and forwards it to the cloud server. The XR terminal device collects local motion, posture and other information and uploads it to the enhanced CPE device. Receive the streaming video data sent from the enhanced CPE, and process and display it. The specific implementation steps are as follows:

[0086] The cloud server runs the XR server application, and the enhanced CPE runs the XR client application;

[0087] The XR client (set up in the enhanced CPE) accesses the XR server according to the network address and establishes a server-client network connection;

[0088] The XR terminal device uploads the locally acquired motion, posture, environment, image and other information to the enhanced CPE, and requests the enhanced CPE to send the rendered video data;

[0089] The XR client deployed on the enhanced CPE uploads local motion, posture, environment and other information to the cloud server, and requests the server to send rendered video data;

[0090] The XR server application performs rendering calculations based on the relevant information uploaded by the XR client, and sends the rendered video data to the client;

[0091] The XR client deployed on the enhanced CPE receives the video data sent by the cloud server and processes it, such as overlaying background images, and multicasts the video data to multiple XR devices in the local area network.

[0092] After the CPE device performs the multi-device data transmission operation, the rendered image data of the XR application can be received by multiple XR devices. This solves the problem that other participants cannot watch XR-related content because the XR server-client can only connect one-to-one.

[0093] Step 104: If the components are a third CPE device and a third XR terminal device, the third XR terminal device performs a rendering calculation operation; wherein the third CPE device is provided with a server, and the third XR terminal device is provided with a client.

[0094] In this embodiment, the rendering calculation operation includes:

[0095] The server corresponding to the third CPE device receives the third operation data transmitted by the client corresponding to each of the third XR terminal devices;

[0096] The server corresponding to the third CPE device renders each of the third operation data to obtain third rendered video data corresponding to each of the third operation data;

[0097] The server corresponding to the third CPE device transmits each of the third rendered video data to the corresponding third XR terminal device, so that the third XR terminal device displays according to the third rendered video data.

[0098] In a specific embodiment, an XR server application is deployed at an enhanced CPE, and the XR application deployed at the enhanced CPE performs rendering calculations based on the calculation reference information such as terminal motion and posture sent by the XR client. The rendered video data is sent to the XR client in a streaming manner. The enhanced CPE is connected to the XR terminal device via a local area network. The XR terminal device deploys an XR client application and uploads the motion, environment and other information of the terminal device to the enhanced CPE. The XR terminal device receives the streaming video data sent from the server, and processes and displays it. The specific implementation steps are as follows:

[0099] The enhanced CPE runs the XR server application, and the XR terminal runs the XR client application;

[0100] The XR client accesses the XR server based on the LAN address and establishes a network connection between the server and the client;

[0101] The XR client uploads local motion, posture, environment and other information to the server, and requests the server to send rendered video data;

[0102] The XR server application performs rendering calculations based on the relevant information uploaded by the XR client, and sends the rendered video data to the client;

[0103] The client receives the video data sent by the server and displays it.

[0104] In this solution, rendering calculations are implemented by enhanced CPE without connecting to cloud servers. This solution can provide a more stable XR streaming experience with lower latency.

[0105] See also Figure 2 , Figure 2 It is a structural diagram of a CPE-based XR data processing device provided by an embodiment of the present invention, comprising: a data judgment module 201, a first processing module 202, a second processing module 203 and a third processing module 204;

[0106] The data judgment module is used to judge the components of the XR system;

[0107] The first processing module is configured to, if the components are a first cloud server, a first CPE device and a first XR terminal device, the first CPE device performing a multi-terminal data transmission operation or a video multicast operation; wherein the first cloud server is provided with a server, and the first XR terminal device is provided with a client;

[0108] The second processing module is used for, if the component is a second cloud server and a second CPE device, the second CPE device performs a multi-device data transmission operation; wherein the second cloud server is provided with a server, and the second CPE device is provided with a client;

[0109] The third processing module is used for, if the components are a third CPE device and a third XR terminal device, the third XR terminal device performs rendering calculation operations; wherein, the third CPE device is provided with a server, and the third XR terminal device is provided with a client.

[0110] As an improvement of the above solution, the first CPE device performs a multi-terminal data transmission operation or a video multicast operation, including:

[0111] Receiving video data transmitted by the first cloud server;

[0112] Making a judgment on the video data;

[0113] If the type of the video data is a rendered video, a multi-end data transmission operation is performed;

[0114] If the type of the video data is cached video, a video multicast operation is performed.

[0115] It can be understood that the above-mentioned system item embodiment corresponds to the method item embodiment of the present invention, which can implement the wireless switching control method based on the two-dimensional secret code provided by any one of the above-mentioned method item embodiments of the present invention.

[0116] This embodiment determines the composition of the XR system components. If the components are a first cloud server, a first CPE device and a first XR terminal device, the first CPE device performs a multi-terminal data transmission operation or a video multicast operation; if the components are a second cloud server and a second CPE device, the second CPE device performs a multi-device data transmission operation; if the components are a third CPE device and a third XR terminal device, the third XR terminal device performs a rendering calculation operation. The present invention identifies the composition of the XR system components and analyzes the application scenarios of the XR system, thereby performing XR data transmission for clients and servers with different settings, breaking through the usage restrictions of XR streaming applications with CPE devices and reducing the hardware cost of terminal devices.

[0117] Embodiment 2

[0118] See also Figure 3 , Figure 3 It is a schematic diagram of the structure of a terminal device provided in one embodiment of the present invention.

[0119] A terminal device of this embodiment includes: a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program, the steps of the above-mentioned CPE-based XR data processing method in the embodiment are implemented, for example: Figure 1 Alternatively, when the processor executes the computer program, the functions of each module in the above-mentioned device embodiments are implemented, for example: Figure 2 All modules of the CPE-based XR data processing device are shown.

[0120] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the CPE-based XR data processing method as described in any of the above embodiments.

[0121] Those skilled in the art will understand that the schematic diagram is merely an example of a terminal device and does not constitute a limitation on the terminal device. The terminal device may include more or fewer components than shown in the diagram, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0122] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 301 is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0123] The memory 302 can be used to store the computer program and / or module. The processor 301 implements various functions of the terminal device by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory 302. The memory 302 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0124] Wherein, if the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0125] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0126] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A CPE-based XR data processing method, characterized in that: include: Make judgments about the components of the XR system; If the components are a first cloud server, a first CPE device and a first XR terminal device, the first CPE device performs a multi-terminal data transmission operation or a video multicast operation; wherein the first cloud server is provided with a server, and the first XR terminal device is provided with a client; the XR video data is stored in the first cloud server, the first CPE device is connected to the cloud server via a wide area network, and is connected to the first XR terminal device via a local area network; the XR video data is sent from the first cloud server to the first CPE device, and the first CPE device multicasts the cached XR video data to the XR terminal device connected to the local area network; the specific implementation steps are as follows: the first XR terminal device sends a video data sending request; the first cloud server sends the XR video data to the first CPE device; the first CPE device sends the XR video obtained from the first cloud server to each first XR terminal device in the form of multicast; each first XR terminal device obtains the video data sent by the first CPE device and processes and displays it; If the component is a second cloud server and a second CPE device, the second CPE device performs a multi-device data transmission operation; wherein the second cloud server is provided with a server, and the second CPE device is provided with a client; If the components are a third CPE device and a third XR terminal device, the third XR terminal device performs rendering calculation operations; wherein the third CPE device is provided with a server, and the third XR terminal device is provided with a client.

2. The CPE-based XR data processing method according to claim 1, characterized in that: The first CPE device performs a multi-terminal data transmission operation or a video multicast operation, including: Receiving video data transmitted by the first cloud server; Making a judgment on the video data; If the type of the video data is a rendered video, a multi-end data transmission operation is performed; If the type of the video data is cached video, a video multicast operation is performed.

3. The CPE-based XR data processing method according to claim 2, characterized in that: The multi-terminal data transmission operation includes: The first CPE device generates a communication channel so that the first cloud server is connected to several first XR terminal devices respectively through the communication channel; wherein, the server corresponding to the first cloud server renders the first operation data transmitted by the client corresponding to each first XR terminal device, generates first rendered video data corresponding to each first operation data, and transmits each first rendered video data to the corresponding first XR terminal device; the client corresponding to each first XR terminal device displays according to the first rendered video data.

4. The CPE-based XR data processing method according to claim 2, characterized in that: The video multicast operation includes: The first CPE device receives cached video data transmitted by the first cloud server; The first CPE device transmits the cached video data to a plurality of the first XR terminal devices in a multicast manner, so that the first XR terminal devices display according to the cached video data.

5. The CPE-based XR data processing method according to claim 1, characterized in that: The multi-device data transmission operation includes: The client corresponding to the second CPE device receives second operation data transmitted by a plurality of second XR terminal devices; The client corresponding to the second CPE device transmits the plurality of second operation data to the second cloud server, so that the second cloud server renders each of the second operation data to obtain second rendered video data corresponding to each of the second operation data; The client corresponding to the second CPE device receives a plurality of the second rendered video data, and transmits each second rendered video data to the corresponding second XR terminal device, so that the second XR terminal device displays according to the second rendered video data.

6. The CPE-based XR data processing method according to claim 1, characterized in that: The rendering calculation operation includes: The server corresponding to the third CPE device receives the third operation data transmitted by the client corresponding to each of the third XR terminal devices; The server corresponding to the third CPE device renders each of the third operation data to obtain third rendered video data corresponding to each of the third operation data; The server corresponding to the third CPE device transmits each of the third rendered video data to the corresponding third XR terminal device, so that the third XR terminal device displays according to the third rendered video data.

7. A CPE-based XR data processing device, characterized in that: include: A data judgment module, a first processing module, a second processing module and a third processing module; The data judgment module is used to judge the components of the XR system; The first processing module is used for, if the components are a first cloud server, a first CPE device and a first XR terminal device, the first CPE device performs a multi-terminal data transmission operation or a video multicast operation; wherein the first cloud server is provided with a server, and the first XR terminal device is provided with a client; the XR video data is stored in the first cloud server, the first CPE device is connected to the cloud server via a wide area network, and is connected to the first XR terminal device via a local area network; the XR video data is sent from the first cloud server to the first CPE device, and the first CPE device multicasts the cached XR video data to the XR terminal device connected to the local area network; the specific implementation steps are as follows: the first XR terminal device sends a video data sending request; the first cloud server sends the XR video data to the first CPE device; the first CPE device sends the XR video obtained from the first cloud server to each first XR terminal device in the form of multicast; each first XR terminal device obtains the video data sent by the first CPE device and processes and displays it; The second processing module is used for, if the component is a second cloud server and a second CPE device, the second CPE device performs a multi-device data transmission operation; wherein the second cloud server is provided with a server, and the second CPE device is provided with a client; The third processing module is used for, if the components are a third CPE device and a third XR terminal device, the third XR terminal device performs rendering calculation operations; wherein, the third CPE device is provided with a server, and the third XR terminal device is provided with a client.

8. The CPE-based XR data processing device according to claim 7, characterized in that: The first CPE device performs a multi-terminal data transmission operation or a video multicast operation, including: Receiving video data transmitted by the first cloud server; Making a judgment on the video data; If the type of the video data is a rendered video, a multi-end data transmission operation is performed; If the type of the video data is cached video, a video multicast operation is performed.

9. A computer terminal device, characterized in that: It comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements a CPE-based XR data processing method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute a CPE-based XR data processing method as described in any one of claims 1 to 6.