Extended reality data rendering method and system

Abstract

The embodiment of the specification provides an extended reality data rendering method and system. The extended reality display device comprises a local extended reality application and a stream data receiving end. The stream data receiving end is configured with a data receiving module corresponding to at least one data transmission mode. The extended reality data rendering method comprises the following steps: the local extended reality application receives a viewing operation and generates a starting instruction according to the viewing operation, and sends the starting instruction to the stream data receiving end; the stream data receiving end determines a target transmission mode from the at least one data transmission mode in response to the starting instruction, receives stream data transmitted by a data sending end through the data receiving module corresponding to the target transmission mode according to a data receiving process corresponding to the target transmission mode, and performs rendering and display. Seamless switching from the local extended reality application to the stream data receiving end is realized, and the stream data receiving end can simultaneously support multiple transmission modes.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of computer technology, and particularly relate to an extended reality data rendering method and system. BACKGROUND

[0002] With the rapid development of computer technology and Internet technology, the extended reality technology is born accordingly. The extended reality technology refers to the fusion of virtual content and real scenes by using hardware devices combined with various technical means, including AR (Augmented Reality), VR (Virtual Reality) and MR (Mixed Reality). With the development of extended reality technology, there may be multiple interactive scenarios for extended reality devices, which need to render and switch different scene contents for display to users. However, due to the performance limitations of the current extended reality display devices, complex rendering programs cannot be run. SUMMARY

[0003] Therefore, the embodiments of the present specification provide an extended reality data rendering method and system, and a commodity data rendering method to solve the technical defects in the prior art.

[0004] According to a first aspect of the embodiments of the present specification, an extended reality data rendering method is provided, applied to an extended reality display device, the extended reality display device comprising a local extended reality application and a stream data receiving end, the stream data receiving end being configured with a data receiving module corresponding to at least one data transmission mode, and the extended reality data rendering method comprising:

[0005] The local extended reality application receives a viewing operation and generates a start instruction according to the viewing operation, and sends the start instruction to the stream data receiving end;

[0006] The stream data receiving end determines a target transmission mode from the at least one data transmission mode in response to the start instruction; receives stream data transmitted by a data sending end through the data receiving module corresponding to the target transmission mode according to a data receiving process corresponding to the target transmission mode, and performs rendering and display.

[0007] According to a second aspect of the embodiments of the present specification, an extended reality data rendering method is provided, applied to a stream data receiving end in an extended reality display device, the stream data receiving end being configured with a data receiving module corresponding to at least one data transmission mode, and the extended reality data rendering method comprising:

[0008] Receiving a start instruction, wherein the start instruction is generated and sent by a local extended reality application in the extended reality display device based on a detected viewing operation;

[0009] Determining a target transmission mode from the at least one data transmission mode in response to the start instruction;

[0010] The data receiving module corresponding to the target transmission mode receives the streaming data transmitted by the data sending end according to the data receiving process corresponding to the target transmission mode, and renders and displays the streaming data.

[0011] According to a third aspect of the embodiments of the present specification, a method for rendering extended reality data is provided, applied to a local extended reality application in an extended reality display device, and the method comprises:

[0012] A viewing operation is received, and a start instruction is generated according to the viewing operation, wherein the start instruction carries a transmission parameter;

[0013] The start instruction is sent to the streaming data receiving end, wherein the streaming data receiving end is configured with a data receiving module corresponding to at least one data transmission mode, and the transmission parameter carried in the start instruction is used to determine a target transmission mode from the at least one data transmission mode by the streaming data receiving end.

[0014] According to a fourth aspect of the embodiments of the present specification, an extended reality data rendering system is provided, comprising a data sending end and an extended reality display device, the data sending end is provided with a data sending module corresponding to at least one data transmission mode, the extended reality display device is provided with a local extended reality application and a streaming data receiving end, and the streaming data receiving end is configured with a data receiving module corresponding to at least one data transmission mode;

[0015] The data sending end is configured to collect to-be-rendered data, generate streaming data according to the to-be-rendered data, and transmit the streaming data to the streaming data receiving end based on a corresponding target transmission mode;

[0016] The local extended reality application is configured to receive a viewing operation and generate a start instruction according to the viewing operation, and send the start instruction to the streaming data receiving end;

[0017] The streaming data receiving end is configured to determine a target transmission mode from at least one data transmission mode in response to the start instruction; receive the streaming data transmitted by the data sending end through a data receiving module corresponding to the target transmission mode according to a data receiving process corresponding to the target transmission mode, and render and display the streaming data.

[0018] According to a fifth aspect of the embodiments of the present specification, a method for rendering product data is provided, applied to a head-mounted display, the head-mounted display comprising a local product application and a product streaming data receiving end, the product streaming data receiving end being configured with a data receiving module corresponding to at least one data transmission mode, and the product data rendering method comprising:

[0019] The local product application receives a product viewing operation and generates a start instruction according to the product viewing operation, and sends the start instruction to the product streaming data receiving end;

[0020] The commodity stream data receiving end determines a target transmission mode from at least one data transmission mode in response to a start instruction, receives commodity stream data transmitted by a data sending end through a data receiving module corresponding to the target transmission mode according to a data receiving process corresponding to the target transmission mode, and performs rendering and display.

[0021] According to a sixth aspect of an embodiment of the present specification, a computing device is provided, comprising:

[0022] a memory and a processor;

[0023] The memory is configured to store computer-executable instructions, and the processor is configured to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned extended reality data rendering method.

[0024] According to a seventh aspect of an embodiment of the present specification, a computer-readable storage medium is provided, which stores computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned extended reality data rendering method.

[0025] According to an eighth aspect of an embodiment of the present specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to perform the steps of the above-mentioned extended reality data rendering method.

[0026] One embodiment of the present specification provides an extended reality data rendering method applied to an extended reality display device, the extended reality display device comprising a local extended reality application and a stream data receiving end, the stream data receiving end being configured with a data receiving module corresponding to at least one data transmission mode, the extended reality data rendering method comprising: the local extended reality application receiving a viewing operation and generating a start instruction according to the viewing operation, and sending the start instruction to the stream data receiving end; the stream data receiving end determining a target transmission mode from at least one data transmission mode in response to the start instruction, receiving stream data transmitted by a data sending end through a data receiving module corresponding to the target transmission mode according to a data receiving process corresponding to the target transmission mode, and performing rendering and display.

[0027] In this case, a stream data receiving end is newly arranged in the extended reality display device, and the stream data receiving end is configured with a data receiving module corresponding to at least one data transmission mode, that is, the stream data receiving end is a data receiving end common to the at least one data transmission mode and can support the at least one data transmission mode at the same time. When a user triggers a certain viewing operation in the local extended reality application, the local extended reality application can generate a corresponding starting instruction based on the viewing operation, send the starting instruction to the stream data receiving end, and the stream data receiving end can determine the target transmission mode of the required stream data in response to the starting instruction, and then start the transmission process corresponding to the target transmission mode to realize stream data receiving and rendering. In this way, the complex rendering program can be implemented in other data sending ends outside the extended reality display device, and the extended reality display device performs data rendering and display through the stream mode, realizes seamless switching from the local extended reality application to the stream data receiving end, and the stream data receiving end can be compatible with multiple transmission modes at the same time, so as to accurately and efficiently switch different transmission modes, avoid the lag from switching from the local extended reality application to the stream data receiving end, and the lag caused by switching different transmission modes, and greatly improve the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a scene diagram of an extended reality data rendering method provided by an embodiment of the present specification;

[0029] Figure 2 is a flowchart of an extended reality data rendering method provided by an embodiment of the present specification;

[0030] Figure 3a is a scene diagram of PC streaming provided by an embodiment of the present specification;

[0031] Figure 3b is a scene diagram of cloud rendering provided by an embodiment of the present specification;

[0032] Figure 3c is a diagram of application switching in an extended reality display device provided by an embodiment of the present specification;

[0033] Figure 4 is a structural block diagram of an extended reality display device provided by an embodiment of the present specification;

[0034] Figure 5 is a flowchart of another extended reality data rendering method provided by an embodiment of the present specification;

[0035] Figure 6 is a structural block diagram of a stream data receiving end provided by an embodiment of the present specification;

[0036] Figure 7 is a flowchart of yet another extended reality data rendering method provided by an embodiment of the present specification;

[0037] Figure 8 is a structural block diagram of a local extended reality application provided by an embodiment of the present specification;

[0038] Figure 9 is a structural block diagram of an extended reality data rendering system provided by an embodiment of the present specification;

[0039] Figure 10 is a process flowchart of a commodity data rendering method provided by an embodiment of the present specification;

[0040] Figure 11 is a structural block diagram of a computing device provided by an embodiment of the present specification. DETAILED DESCRIPTION

[0041] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present specification. However, the present specification can be practiced without the specific details, other than in the examples, and it can be apparent to those skilled in the art that the present specification can be practiced without incorporating some or all of the specific details and other examples might work in the same manner or achieve the same results. In other instances, details might be known by those skilled in the art and therefore not be discussed in order not to obscure the present specification.

[0042] The terminology used in one or more embodiments of the present specification is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present specification. As used in one or more embodiments of the present specification and the accompanying claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in one or more embodiments of the present specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0043] It will be understood that, although the terms first, second, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used solely to distinguish one from another only. For example, without departing from the scope of one or more embodiments of the present specification, first can be termed second, and similarly, second can be termed first. The term "if' as used herein, can be interpreted as meaning "when" or "in response to determining" depending on the context.

[0044] First, the noun terms related to one or more embodiments of the present specification are explained.

[0045] Extended Reality (XR): refers to the use of hardware devices combined with various technical means (such as computer technology and wearable devices) to fuse virtual content and real scenes, interactive environments, including VR (Virtual Reality), AR (Augmented Reality), MR (Mediated Reality), and other immersive technologies that may appear due to technological advances. Among them, VR (Virtual Reality) is also called virtual technology, virtual environment, which uses computers to simulate a three-dimensional virtual world, provides users with a sense of immersion and immediacy through sensory simulation, and provides users with visual and other sensory simulations, making users feel as if they were there, and can instantly and unlimitedly observe three-dimensional space. Just like taking people to a virtual simulation world; AR (Augmented Reality) is a technology that calculates the position and angle of the camera image in real time and adds corresponding images, which uses various technical means to superimpose computer-generated virtual objects or non-geometric information about real objects onto real-world scenes, thereby enhancing the real world; MR (Mediated Reality) introduces virtual scene information into the real environment, builds an interactive information loop between the real world, the virtual world and the user, and enhances the user's experience of reality, real-time interaction and imagination. It has the characteristics of reality, real-time interaction and imagination.

[0046] XR headset: here refers to XR head-mounted display all-in-one machine, most of which are built-in Android operating system.

[0047] Streaming: transmitting digital audio or video over the Internet, sound and video data will be transmitted to users in the form of data streams. In the embodiments of the present specification, it refers to the technology of transmitting audio and video data to the XR headset for rendering and display through the network or USB (Universal Serial Bus) wired transmission.

[0048] DisplayPort (DisplayPort): a technology for high-speed audio and video data transmission through USB connection cable, which can support full HD, 3D, 4K ultra HD and full screen output, as well as higher resolution and refresh rate under the premise of low power consumption.

[0049] ALVR: an open source wireless streaming framework, its principle is the same as that of various video projection at present, as long as it is in the same WIFI "local area network", data projection can be realized.

[0050] Cloud Rendering: A streaming service sends audio and video, such as games, to an application or browser installed on a receiving device through an internet connection. The game is rendered and played on a remote server, while all content is viewed and interacted with on the local device.

[0051] CloudXR: CloudXR supports immersive AR, MR, or VR experiences through streaming from a server.

[0052] Local XR Application: A local XR application running on a headset device, usually developed as an Android application based on Unity3D or Unreal.

[0053] Local Streaming Application: A receiving application running on a headset device that receives and renders various streaming data, usually an Android application.

[0054] PC: Personal Computer, commonly referred to as PC, refers to computers suitable for personal use in terms of size, performance, and price, and directly controlled by end users. It is composed of hardware system and software system, and is a device that can run independently and complete specific functions. Desktop computers, laptops, tablets, and ultrabooks all belong to the category of personal computers.

[0055] PC Streaming Application: A specific application running on a PC that can capture audio and video data from the XR application running on the PC and send it to the headset, and can also receive user interaction data sent by the headset, usually an executable program for Windows.

[0056] SDK (Software Development Kit): Generally refers to a collection of development tools established by software engineers for specific software packages, software frameworks, hardware platforms, operating systems, etc. Software development tools include a collection of related documents, examples, and tools that assist in the development of a certain type of software. It can simply provide some files for the application programming interface API of a certain programming language, but it may also include complex hardware that can communicate with certain embedded systems. Common tools include utilities for debugging and other purposes. SDK also often includes sample code, supporting technical notes, or other supporting documents to clarify doubts for basic reference materials.

[0057] It should be noted that the current local rendering is developed by Unity3D or Unreal, and a local Android application can be packaged, which is actually an Activity page in Android, which renders 3D through opengl (Open Graphics Library). The streaming page is also an Activity, so it can use opengl to draw the received streaming data to the screen, or directly render the cloud streaming data through the embedded webview method. Therefore, the switching between local and streaming is essentially the jump of the Activity page in different Android applications. Therefore, in the embodiments of the present specification, the streaming scheme on different terminals can be unified to provide seamless application switching and better immersion for users.

[0058] In practical applications, the PC can collect and compress the picture of the extended reality application and / or game, and then send it to the extended reality display device through wired or wireless transmission. In the extended reality display device, different streaming methods are identified, and the corresponding receiving process is started to display audio and video data. Then the extended reality display device can transmit the position information of the head-mounted display, the position and key information of the handle, and other interactive information to the PC or cloud server for interaction.

[0059] Figure 1 A scene diagram of an extended reality data rendering method according to an embodiment of the present specification is shown in FIG. 1. Figure 1 As shown in FIG. 1, the local extended reality application in the extended reality display device can start the streaming data receiving end, the PC can transmit streaming data to the streaming data receiving end through wired transmission and wireless transmission, and the cloud server can transmit streaming data to the streaming data receiving end through cloud rendering. In addition, the streaming data receiving end can return interactive data to the PC and the cloud server through the corresponding transmission method, and can also exit the streaming data receiving end and switch back to the local extended reality application.

[0060] In the present specification, an extended reality data rendering method, system and commodity data rendering method are provided, which are described in detail one by one in the following embodiments.

[0061] Referring to Figure 2 , Figure 2 A flowchart of an extended reality data rendering method according to an embodiment of the present specification is shown in FIG. 1, which is applied to an extended reality display device including a local extended reality application and a streaming data receiving end. The streaming data receiving end is configured with a data receiving module corresponding to at least one data transmission method, and the extended reality data rendering method specifically includes the following steps:

[0062] Step 202: The local extended reality application receives the viewing operation and generates a starting instruction according to the viewing operation, and sends the starting instruction to the stream data receiving end.

[0063] Specifically, the extended reality display device can be an XR head-mounted display, and the extended reality display device is provided with a local extended reality application and a stream data receiving end. The local extended reality application refers to an application in the extended reality display device for user operation to display extended reality functions. The stream data receiving end refers to a receiving end in the extended reality display device for receiving stream data. For example, the local extended reality application is a local XR application in the XR head-mounted display, and the stream data receiving end is a local stream application in the XR head-mounted display.

[0064] In addition, the viewing operation refers to a display operation triggered by the user in the local extended reality application for an object, which is used to indicate that the user wants to view the object in the extended reality display device. For example, the user clicks a clothing show control in the local extended reality application, which triggers a viewing operation for the clothing show to indicate that the user wants to view the related scene picture of the clothing show. Secondly, the starting instruction refers to an instruction for starting the stream data receiving end, i.e. switching from the local extended reality application to the stream data receiving end to show the stream data received by the stream data receiving end to the user.

[0065] Among them, the stream data refers to data that needs to be obtained from other devices by the stream data receiving end to render and display to the user. The stream data can be static pictures, texts, audio and video data, etc.

[0066] It should be noted that the extended reality technology can be applied in various scenes. In each application scenario, there can be multiple interaction scenarios for the extended reality. For example, the application of the extended reality display device in the e-commerce scene, the extended reality display device has many 3D shops, 3D street views, virtual live rooms, clothing show scenes, etc. It may be necessary to transfer in different shops or blocks. For example, the application of the extended reality display device in the office scene, the extended reality display device switches from a single-person office scene to a multi-person conference scene. For example, the application of the extended reality display device in the large music festival scene, the main scene can be a stage with a million people on the screen, and the road to the stage is a simple transportation tool, etc.

[0067] These different interaction scenarios may be developed by different teams, use different technology stacks, and more importantly, have different requirements for the hardware running environment. For example, some interaction scenarios focus on the convenience of user use and consider running the execution program directly on the extended reality display device; while some scenarios focus on display effects and are limited by the insufficient performance of the extended reality display device hardware, and need to run the execution program on a high-performance PC or cloud server, and then render and interact on the extended reality display device through wired, wireless streaming, or cloud streaming.

[0068] However, users do not need to care about whether the scene picture they see is running on an extended reality display device, a PC, or a cloud server, and do not want to see experience interruptions caused by scene switching. Therefore, in actual applications, a streaming data receiving end that can be compatible with multiple streaming schemes is set in the extended reality display device, that is, the streaming data receiving end is configured with at least one data receiving module corresponding to a data transmission mode, and supports at least one data transmission mode, so that the user can seamlessly switch between virtual scenes running on different data sending ends after wearing the extended reality display device.

[0069] It should be noted that the data transmission mode refers to the way in which streaming data is transmitted from the data sending end to the streaming data receiving end. The data sending end can be a PC or a cloud server. When the data sending end is a PC, the data transmission mode can be divided into wireless transmission and wired transmission; when the data sending end is a cloud server, the data transmission mode can be divided into cloud transmission, etc.

[0070] That is, to see a 3D extended reality virtual scene in the extended reality display device, there are mainly the following three ways:

[0071] Local extended reality application: Currently, the mainstream extended reality display device runs the Android operating system, so it can use a 3D engine (such as Unity, Unreal, etc.) to package into an Android platform supported installable file APK (Android application package), and directly install and run on the extended reality display device.

[0072] High-performance PC streaming: PC streaming is divided into wired transmission and wireless transmission. Wired transmission generally transmits streaming data through USB DisplayPort; while wireless transmission is generally developed based on an open source streaming framework, such as ALVR. Both of these two transmission modes need to set a data sending end for sending streaming data on the PC side, which can push the streaming data to be displayed out, while a streaming data receiving end for receiving streaming data is set on the extended reality display device to receive and display the streaming data.

[0073] Cloud rendering: taking the cloud game scene as an example, the server of cloud game generally provides an executable environment in the cloud by the cloud service provider, the cloud rendering application developed by the 3D engine can be run in the cloud server, and the cloud server is responsible for collecting and compressing transmission, and after the stream data receiving end is connected to the cloud server, a WEB-based application program is usually used to display and interact with the game, that is, the stream data is pushed to the stream data receiving end through the cloud transmission.

[0074] In actual application, whether it is PC streaming or cloud rendering, a stream data receiving end that can receive stream data is needed in the extended reality display device, so a general stream data receiving end is developed in the extended reality display device in the embodiment of the present specification, which is compatible with multiple stream data transmission, that is, compatible with multiple data transmission modes, so that the local extended reality application can seamlessly jump to the stream data receiving end, realize the switching of the local extended reality application and the stream data receiving end, and the switching of different transmission modes, avoid stuttering, and improve the user experience.

[0075] In an optional implementation of the embodiment, the viewing operation carries operation object information; the starting instruction is generated according to the viewing operation, including:

[0076] Querying the target transmission mode corresponding to the operation object information carried by the viewing operation;

[0077] Determining the transmission parameter representing the target transmission mode, and generating the starting instruction according to the transmission parameter.

[0078] In actual application, the user wants to watch what content through the extended reality display device, and can trigger the display operation in the local extended reality application for the corresponding object to indicate that the user wants to view the related content in the extended reality display device, so the operation object information can be carried in the viewing operation received by the local extended reality application, at this time, the local extended reality application can query the target transmission mode corresponding to the operation object information carried by the viewing operation, determine the transmission parameter representing the target transmission mode, and generate the starting instruction according to the transmission parameter; That is, the transmission parameter can be carried in the starting instruction generated by the local extended reality application, which can indicate the transmission mode of the stream data required by the stream data receiving end, so that the stream data receiving end can determine the corresponding target transmission mode when receiving the starting instruction, to realize the serial data transmission, such as transmission parameter pc_wireless, indicating wireless transmission; Transmission parameter pc_usb, indicating wired transmission; Transmission parameter cloud+url, indicating cloud transmission. Specifically, the starting instruction is generated according to the transmission parameter, which can carry the transmission parameter in the starting instruction.

[0079] In a specific implementation, the extended reality display device is generally based on an Android system, and thus the stream data receiving end developed in the extended reality display device can also be an Android application. The local extended reality application can start the stream application by using Activity.StartActivity() provided by the Android system, that is, the starting instruction is an invocation instruction of Activity.StartActivity(), and the transmission parameter can be passed in the starting instruction to determine the transmission manner of the stream data to be received.

[0080] It should be noted that the user can select an operation object to be viewed in the local extended reality application, and then initiate a viewing operation for the operation object. The viewing operation carries operation object information. When the local extended reality application receives the viewing operation, the local extended reality application can determine, based on the operation object information, whether the to-be-displayed data corresponding to the operation object information is data in the local extended reality application or stream data, that is, whether the to-be-displayed data corresponding to the operation object information is stream data. If the to-be-displayed data is stream data, the target transmission manner corresponding to the operation object information can be further queried, and then the transmission parameter representing the target transmission manner is carried in the starting instruction and sent to the stream data receiving end, so as to start the stream data receiving end and inform the data transmission manner of the required stream data.

[0081] For example, in a scene provided by the local extended reality application of the user, the user sees a store and clicks the door of the store to indicate that the user wants to enter the store. It is assumed that the related to-be-displayed data of the store needs to be obtained from another device. At this time, the target transmission manner corresponding to the store can be queried as wireless transmission. At this time, a starting instruction carrying the transmission parameter pc_wireless can be sent to the stream data receiving end, so as to start the stream data receiving end in the extended reality display device.

[0082] In the embodiments of the present specification, the user uses the extended reality display device in the local extended reality application. Through the operation in the local extended reality application, the stream data receiving end can be started, and the data transmission manner of the required stream data can be passed to the stream data receiving end. The stream data receiving end subsequently receives the stream data in the corresponding transmission manner, renders and displays the obtained stream data to the user, and improves the user experience.

[0083] In an optional implementation of the present embodiment, the target transmission manner corresponding to the operation object information carried in the viewing operation is queried, including:

[0084] The corresponding relationship between the local query object and the data transmission manner is determined to determine the target transmission manner corresponding to the operation object information carried in the viewing operation; or,

[0085] The operation object information is uploaded to a cloud storage server, and a target transmission mode returned by the cloud storage server is received, wherein the cloud storage server stores data transmission modes corresponding to respective objects.

[0086] It should be noted that each object in the extended reality display device can be pre-configured with a corresponding data transmission mode, that is, different transmission modes such as wired transmission, wireless transmission, and cloud transmission can be set for corresponding objects in the extended reality display device scene during development to obtain a correspondence between the object and the data transmission mode. The correspondence between the object and the data transmission mode can be stored locally in the local extended reality application or in the cloud storage server.

[0087] In actual application, the target transmission mode corresponding to the operation object information carried by the query and viewing operation includes two possible implementation manners. One possible implementation manner is that the local extended reality application locally queries the correspondence between the object and the data transmission mode to determine the target transmission mode corresponding to the operation object information carried by the viewing operation. Another possible implementation manner is that the local extended reality application uploads the operation object information to a cloud storage server and receives a target transmission mode returned by the cloud storage server, wherein the cloud storage server stores data transmission modes corresponding to respective objects. In this way, the query flexibility of the target transmission mode is improved, and the adaptability is higher.

[0088] For example, a user is “roaming” in a local extended reality application and sees a show. Assuming that the correspondence between the object and the data transmission mode is stored locally in the local extended reality application, the local extended reality application can locally query that the show corresponds to wired transmission. Alternatively, assuming that the correspondence between the object and the data transmission mode is stored in a cloud storage server, after the user clicks the 3D model of the show, the local extended reality application can query that the show corresponds to wired transmission from the cloud storage server through an http protocol. Thus, the user needs to obtain streaming data through a wired transmission mode, that is, wired streaming data, after clicking the 3D model of the show in the local extended reality application.

[0089] In the embodiments of the present specification, a general streaming data receiving end is developed in the extended reality display device, which is compatible with multiple transmission modes of streaming data, that is, compatible with multiple data transmission modes, so that the local extended reality application can seamlessly jump to the streaming data receiving end, realize switching between the local extended reality application and the streaming data receiving end, and switching between different transmission modes, avoid stuttering, and improve user experience.

[0090] Step 204: In response to the starting instruction, the stream data receiving end determines a target transmission mode from at least one data transmission mode, receives the stream data transmitted by the data sending end through the data receiving module corresponding to the target transmission mode according to the data receiving process corresponding to the target transmission mode, and performs rendering display.

[0091] It should be noted that different data transmission modes need to be supported by different modules in the stream data receiving end, such as a wireless data receiving module for wireless transmission mode, a wired data receiving module for wired transmission mode, and a cloud rendering module for cloud transmission mode. Therefore, after receiving the starting instruction sent by the local extended reality application, the stream data receiving end can respond to the starting instruction, determine a target transmission mode from at least one data transmission mode, and then receive the stream data transmitted by the data sending end through the data receiving module corresponding to the target transmission mode according to the data receiving process corresponding to the target transmission mode, and perform rendering display.

[0092] In an optional implementation of the present embodiment, determining a target transmission mode from at least one data transmission mode includes:

[0093] In the case where the starting instruction carries transmission parameters, the corresponding target transmission mode is determined according to the transmission parameters carried in the starting instruction; or,

[0094] Listening to at least one data transmission mode, determining a target transmission mode that meets the transmission condition.

[0095] In a possible implementation, if the starting instruction sent by the local extended reality application to the stream data receiving end carries transmission parameters, since the transmission parameters can indicate the corresponding transmission mode, the stream data receiving end can directly determine the corresponding target transmission mode according to the transmission parameters carried in the starting instruction after receiving the starting instruction. The transmission parameters carried in the starting instruction refer to parameters that can indicate the transmission mode of the stream data required by the stream data receiving end, such as transmission parameter pc_wireless indicating wireless transmission, transmission parameter pc_usb indicating wired transmission, and transmission parameter cloud+url indicating cloud transmission.

[0096] For example, the transmission parameter received by the stream data receiving end is pc_wireless, the target transmission mode is wireless transmission mode, i.e., PC wireless streaming, the data listening service of the ALVR framework is started at this time; the transmission parameter received by the stream data receiving end is pc_usb, the target transmission mode is wired transmission mode, i.e., PC wired streaming, USB data transmission is listened to at this time; the transmission parameter received by the stream data receiving end is cloud+url, i.e., cloud rendering, a WEB application (browser) can be directly opened through the URL parameter to receive at this time.

[0097] In another possible implementation, the start instruction sent by the local extended reality application to the stream data receiving end can also not carry the transmission parameter, and the stream data receiving end autonomously determines the target transmission mode used to receive the corresponding stream data after receiving the start instruction. In a specific implementation, the stream data receiving end can listen to at least one data transmission mode, and determine a target transmission mode that meets a transmission condition. The transmission condition is a condition that needs to be met for the data transmission mode to achieve data transmission. For example, the transmission condition corresponding to wired transmission is that a connection line exists between the local extended reality application and the stream data receiving end, that is, the local extended reality application and the stream data receiving end are connected by wire; the transmission condition corresponding to wireless transmission is that the local extended reality application and the stream data receiving end are in the same local area network, that is, the local extended reality application and the stream data receiving end are connected by wire; and the transmission condition corresponding to cloud transmission is that the corresponding stream server is started.

[0098] In actual application, after receiving the start instruction sent by the local extended reality application, the stream data receiving end can listen to whether a connection line exists between the local extended reality application and the stream data receiving end, whether they are in the same local area network, and whether the corresponding stream server is started. If a transmission mode that meets the pre-set transmission condition is listened to, the transmission mode can be used as the target transmission mode.

[0099] In addition, if at least two data transmission modes meet the corresponding transmission conditions, the at least two data transmission modes can be used as the target transmission mode, that is, the stream data is received through multiple transmission modes; or, one of the two data transmission modes can be randomly selected as the target transmission mode; or, the priority of different data transmission modes can be pre-set, and the final target transmission mode can be determined from the at least two data transmission modes based on the priority. Furthermore, if none of the data transmission modes meets the corresponding transmission condition, an error prompt can be given.

[0100] In the embodiments of the present application, the start instruction sent by the local extended reality application to the stream data receiving end can carry the transmission parameter indicating the transmission mode, or can not carry the transmission parameter, that is, the local extended reality application can specify the transmission mode of the stream data, or the stream data receiving end can automatically determine the transmission mode of the stream data, greatly improving the flexibility of determining the target transmission mode and better adapting to different application scenarios.

[0101] In an optional implementation of the embodiment, the target transmission manner is wireless transmission, the data receiving module corresponding to the wireless transmission is a wireless data receiving module, and the data sending end and the extended reality display device are in the same local area network; the stream data transmitted by the data sending end is received according to the data receiving process corresponding to the target transmission manner through the data receiving module corresponding to the target transmission manner, including:

[0102] The wireless channel between the extended reality display device and the data sending end is listened to through the wireless data receiving module.

[0103] The stream data transmitted by the data sending end through the wireless channel is received.

[0104] Specifically, the target transmission manner is wireless transmission, that is, wireless streaming, and the data receiving module corresponding to the wireless transmission is a wireless data receiving module. It should be noted that the wireless transmission manner is generally for a PC end, that is, the data sending end is arranged in the PC end, the PC end is wirelessly streamed, and the PC end and the extended reality display device are required to be in the same local area network and communicate through a wireless transmission protocol.

[0105] In actual application, a wireless streaming framework ALVR can be used to develop a wireless data receiving module in the stream data receiving end of the extended reality display device, so as to receive the stream data transmitted by the data sending end in the PC end through the wireless transmission manner. The ALVR provides two sets of SDKs in the form of source code for the PC end and the stream data receiving end, that is, one set of SDK for sending stream data is arranged in the PC end, and one set of SDK for receiving stream data is arranged in the stream data receiving end. A data sending end supporting wireless transmission is developed in the PC end to collect and render pictures and send compressed stream data to the stream data receiving end of the extended reality display device, and a wireless data receiving module is developed in the stream data receiving end, which is mainly used to receive the stream data transmitted by the data sending end through the wireless transmission manner.

[0106] In the embodiment of the present specification, under the wireless transmission manner, the stream data receiving end can listen to the wireless channel between the extended reality display device and the data sending end through the wireless data receiving module, and receive the stream data transmitted by the data sending end through the wireless channel. In this way, the data sending module supporting wireless transmission is arranged in the PC end, and the wireless data receiving module supporting wireless transmission is arranged in the stream data receiving end, so as to realize wireless streaming from the PC end to the extended reality display device, seamlessly switch from the local extended reality application to the stream data receiving end, receive the stream data in the wireless transmission manner and render and display, avoid lag, and improve user experience.

[0107] In an optional implementation of the embodiment, the target transmission mode is wired transmission, and the data receiving module corresponding to the wired transmission is a wired data receiving module; the streaming data transmitted by the data sending end is received through the data receiving module corresponding to the target transmission mode according to the data receiving process corresponding to the target transmission mode, including:

[0108] determining whether a connection line exists between the extended reality display device and the data sending end;

[0109] In the case where the connection line exists, the wired data receiving module is used to monitor the connection line between the extended reality display device and the data sending end, and receive the streaming data transmitted through the connection line.

[0110] Specifically, the target transmission mode can also be wired transmission, that is, wired streaming, and the data receiving module corresponding to the wired transmission is a wired data receiving module. It should be noted that the wired transmission mode is generally also for the PC end, that is, the data sending end is arranged in the PC end, and the PC end wired streaming requires the PC end and the extended reality display device to be connected through a connection line. For example, the PC end and the extended reality display device can be connected through a USB (Universal Serial Bus) data connection line, and the streaming data is transmitted through a standard DispalyPort, that is, USB wired streaming.

[0111] In actual application, similar to the PC wireless streaming, a data sending end supporting wired transmission needs to be developed in the PC end to collect the rendered picture and send the compressed streaming data to the streaming data receiving end of the extended reality display device, and a wired data receiving module is developed in the streaming data receiving end, which is mainly used to receive the streaming data transmitted by the data sending end through the wired transmission mode.

[0112] In the embodiment of the present specification, in the wired transmission mode, the streaming data receiving end can determine whether a connection line exists between the extended reality display device and the data sending end, and whether the PC end and the extended reality display device are successfully connected through the connection line, and in the case where the connection line exists, the wired data receiving module is used to monitor the connection line between the extended reality display device and the data sending end, and receive the streaming data transmitted through the connection line. In this way, the data sending module capable of supporting wired transmission is arranged in the PC end, and the wired data receiving module capable of supporting wired transmission is arranged in the streaming data receiving end, so as to realize wired streaming from the PC end to the extended reality display device, seamlessly switch from the local extended reality application to the streaming data receiving end, receive the streaming data in the wired transmission mode and render and display, avoid lag, and improve the user experience.

[0113] It should be noted that the PC terminal and the extended reality display device can also support wireless transmission and wired transmission at the same time, and the data sending end supporting wired transmission and wireless transmission in the PC terminal can be an application program, but different modules are set in the application program to realize wired transmission and wireless transmission; and the data receiving end supporting wireless transmission and wired transmission in the extended reality display device can also be the same application program, that is, the stream data receiving end, and different modules are set in the stream data receiving end to realize wireless transmission and wired transmission at the same time. In this way, a general stream data receiving end is developed in the extended reality display device, which is compatible with multiple stream data transmission, that is, compatible with multiple data transmission modes, so that the local extended reality application can seamlessly jump to the stream data receiving end, realize the switching of the local extended reality application and the stream data receiving end, and the switching of different transmission modes, avoid stuttering, and improve the user experience.

[0114] In an optional embodiment of the present embodiment, the target transmission mode is PC terminal transmission, and the data receiving module corresponding to the PC terminal transmission is a wireless data receiving module and a wired data receiving module. The stream data transmitted by the data sending end is received through the data receiving module corresponding to the target transmission mode according to the data receiving process corresponding to the target transmission mode, including:

[0115] The wireless channel between the extended reality display device and the data sending end is listened to through the wireless data receiving module, and the connection line between the extended reality display device and the data sending end is listened to through the wired data receiving module.

[0116] In the case that the stream data is listened to through the wireless channel, it is determined that the current transmission mode is wireless transmission, and the subsequent data transmission process of wireless transmission is adopted for data interaction; in the case that the stream data is listened to through the connection line, it is determined that the current transmission mode is wired transmission, and the subsequent data transmission process of wired transmission is adopted for data interaction.

[0117] It should be noted that the transmission parameter transmitted by the local extended reality application to the stream data receiving end can not indicate a specific transmission mode, but only indicate the data sending end, such as determining that the target transmission mode is PC terminal transmission based on the transmission parameter. Since the PC terminal transmission can be wireless transmission or wired transmission, the stream data receiving end can listen to the wireless channel and the connection line at the same time at this time to distinguish whether the stream is wired transmission or wireless transmission according to which channel has stream data, thereby improving flexibility and applicability.

[0118] An example is Figure 3a is a scene schematic diagram of PC streaming provided by an embodiment of the present specification, such as Figure 3aAs shown in the example of PC streaming, the 3D store and the 2.5D store are local extended reality applications (i.e., local XR applications) directly running on the extended reality display device, and the clothing show is an application running on the PC to display the special effects of the clothing. The user first enters the 3D or 2.5D store locally on the extended reality display device, clicks on the show of a clothing item in the store, and the local extended reality application starts the streaming data receiving end. The streaming data receiving end can receive the audio and video data of the clothing show scene sent from the PC through wired transmission or wireless transmission. For the user, he does not feel that the picture he sees is running on the extended reality display device or is streamed from the PC, but only feels that he jumps from one XR scene to another.

[0119] In an optional embodiment of the present embodiment, the target transmission mode is cloud transmission, and the data receiving module corresponding to the cloud transmission is a cloud rendering module. The streaming data sent by the data sending end is received through the data receiving module corresponding to the target transmission mode according to the data receiving process corresponding to the target transmission mode, including:

[0120] The streaming server address is set through the cloud rendering module, and the streaming data transmitted by the cloud server corresponding to the streaming server address is received.

[0121] Specifically, the target transmission mode can also be cloud transmission, i.e., cloud rendering streaming, and the required data runs in the cloud. The data receiving module corresponding to the cloud transmission is a cloud rendering module. It should be noted that cloud rendering provides an extended reality user with a choice of integrating the high performance of a cloud server and the mobility of a wireless integrated extended reality display device.

[0122] It should be noted that the streaming data receiving end in the extended reality display device can be rendered locally by integrating the SDK (i.e., cloud rendering module) corresponding to the cloud rendering. When jumping is needed, the streaming data receiving end can be rendered by opening a URL link.

[0123] In practical applications, cloud rendering is to package the XR or 3D application into an executable file (*.exe) and then run it on a cloud server. After the streaming data receiving end integrates the corresponding SDK, the streaming data of the cloud server can be received by setting the streaming server address. In addition, in addition to setting the streaming server address, authentication information can also be set.

[0124] As shown in the example, Figure 3b is a scene diagram of cloud rendering provided by an embodiment of the present specification, as Figure 3b shown, the cloud server renders and collects the required audio and video data, and transmits the compressed audio and video stream to the streaming data receiving end in the extended reality display device.

[0125] In the cloud transmission mode in the embodiments of the present specification, the stream data receiving end can set the stream server address through the cloud rendering module, and then receive the stream data transmitted by the cloud server corresponding to the stream server address. In this way, the cloud rendering module capable of supporting cloud transmission is set in the stream data receiving end, so as to realize cloud rendering streaming, seamlessly switch the local extended reality application to the stream data receiving end, receive the stream data in the cloud rendering mode and render and display, avoid stuttering, and improve the user experience.

[0126] It should be noted that the extended reality display device can simultaneously support wireless transmission, wired transmission and cloud transmission, and the data receiving end supporting the three transmission modes can be the same application program, that is, the stream data receiving end, which is provided with different modules, such as a wireless data receiving module, a wired data receiving module and a cloud rendering module. In this way, a general stream data receiving end is developed in the extended reality display device, which is compatible with multiple stream data transmission, that is, compatible with multiple data transmission modes, so that the local extended reality application can seamlessly jump to the stream data receiving end, realize the switching of the local extended reality application and the stream data receiving end, and the switching of different transmission modes, avoid stuttering, and improve the user experience.

[0127] In an optional embodiment of the present embodiment, after receiving the stream data transmitted by the data sending end and rendering and displaying, the method further comprises:

[0128] Collecting control information of the extended reality display device and / or the control device;

[0129] According to the target transmission mode, the control information is fed back to the data sending end.

[0130] Specifically, the control device can be a device capable of controlling the extended reality display device, that is, the peripheral of the extended reality display device, such as a handle, a control display screen, etc.; the control information refers to the interactive information fed back by the user through the extended reality display device and / or the control device, such as the user can interact with the model in the scene by rotating the extended reality display device and the handle button.

[0131] It should be noted that in addition to receiving the stream data transmitted by other data sending ends, the stream data receiving end can also feed back the control information of the extended reality display device and / or the control device to the data sending end, that is, after the stream data receiving end receives the stream data transmitted by the data sending end and renders and displays it to the user, the user can operate the extended reality display device and / or the control device to interact with the data sending end, feed back the interactive information to the data sending end, so that the data sending end can execute corresponding processing based on the interactive information fed back by the user, such as generating corresponding stream data and transmitting it to the stream data receiving end.

[0132] In actual applications, the receiving end of the streaming data receives the streaming data in different transmission modes, and returns the control information to the data sending end in different modes, that is, the receiving end of the streaming data can collect the control information of the extended reality display device and / or the control device, and then feeds back the control information to the data sending end according to the target transmission mode, that is, the transmission mode of the receiving end of the streaming data. For example, if the receiving end of the streaming data receives the wireless streaming transmitted by the PC end in a wireless manner, the control information can be fed back to the PC end in a wireless manner; if the receiving end of the streaming data receives the wired streaming transmitted by the PC end in a wired manner, the control information can be fed back to the PC end in a wired manner; if the receiving end of the streaming data receives the cloud rendering streaming transmitted by the cloud server in a cloud rendering manner, the control information can be fed back to the cloud server in a cloud rendering manner.

[0133] As shown in the above example, Figure 3b The receiving end of the streaming data in the extended reality display device can feed back the control information to the cloud server.

[0134] In the embodiments of the present specification, the user can operate the extended reality display device and / or the control device for interaction, so that the receiving end of the streaming data can collect the control information of the extended reality display device and / or the control device, and then feed back the control information to the data sending end according to the target transmission mode. In this way, the user can interact with the extended reality display device, and the interactivity is improved.

[0135] In an optional implementation of the present embodiment, after receiving the streaming data transmitted by the data sending end and rendering and displaying, the method further comprises:

[0136] In the case of detecting the streaming exit instruction, terminating the receiving of the streaming data;

[0137] Calling the scheduling interface of the local extended reality application, and returning the local extended reality application.

[0138] It should be noted that when it is necessary to switch back from the receiving end of the streaming data to the local extended reality application, the scheduling interface of the local extended reality application can be called to return the local extended reality application.

[0139] In actual applications, when it is necessary to exit the receiving end of the streaming data, the receiving of the streaming data can be terminated, and the scheduling interface API: Activity.finish() of the local extended reality application based on the application stack of itself will return to the previous local extended reality application for continuous display, which is simple and convenient, and can realize seamless switching between the local extended reality application and the receiving end of the streaming data.

[0140] In the example, Figure 3cis a schematic diagram of application switching in an extended reality display device provided by an embodiment of the present specification, as shown Figure 3c The local extended reality application can start the stream data receiver by StartActivity(), and pass the transmission parameter to the stream data receiver. If the transmission parameter received by the stream data receiver is pc_wireless, the stream data receiver receives PC wireless streaming; if the transmission parameter received by the stream data receiver is pc_usb, the stream data receiver receives PC wired streaming; if the transmission parameter received by the stream data receiver is cloud+url, the stream data receiver receives cloud rendering streaming. When it is necessary to exit the stream data receiver, Activity.finish() can be called to switch the real local extended reality application.

[0141] The extended reality data rendering method provided by the embodiment of the present specification adds a stream data receiver in the extended reality display device, and the stream data receiver is configured with a data receiving module corresponding to at least one data transmission mode, i.e., the stream data receiver is a data receiver common to at least one data transmission mode, which can support the at least one data transmission mode at the same time. When a user triggers a certain viewing operation in the local extended reality application, the local extended reality application can generate a corresponding start instruction based on the viewing operation, send the start instruction to the stream data receiver, and the stream data receiver determines the target transmission mode of the required stream data from the at least one data transmission mode in response to the start instruction, and then starts the transmission process corresponding to the target transmission mode, to realize stream data receiving and rendering. In this way, complex rendering programs can be implemented in other data senders outside the extended reality display device, and the extended reality display device performs data rendering and display through streaming, realizing seamless switching from the local extended reality application to the stream data receiver, and the stream data receiver can be compatible with multiple transmission modes at the same time, so as to accurately and efficiently switch different transmission modes, avoiding the lag from switching from the local extended reality application to the stream data receiver, and the lag caused by switching different transmission modes, greatly improving the user experience.

[0142] Referring to Figure 4 , Figure 4 a structural block diagram of an extended reality display device provided by an embodiment of the present specification is shown, which includes a local extended reality application 402 and a stream data receiver 404, and the stream data receiver 404 is configured with a data receiving module corresponding to at least one data transmission mode;

[0143] The local extended reality application 402 is configured to receive a viewing operation and generate a start instruction according to the viewing operation, and send the start instruction to the stream data receiver;

[0144] The stream data receiving end 404 is configured to determine a target transmission mode from at least one data transmission mode in response to a start instruction; receive stream data transmitted by the data sending end through a data receiving module corresponding to the target transmission mode according to a data receiving process corresponding to the target transmission mode, and perform rendering display.

[0145] Optionally, the stream data receiving end 404 is further configured to:

[0146] In the case that the start instruction carries transmission parameters, the corresponding target transmission mode is determined according to the transmission parameters carried in the start instruction; or,

[0147] The target transmission mode that meets the transmission condition is determined by listening to at least one data transmission mode.

[0148] Optionally, the viewing operation carries operation object information; the local extended reality application 402 is further configured to:

[0149] Query the target transmission mode corresponding to the operation object information carried by the viewing operation;

[0150] Determine the transmission parameters representing the target transmission mode, and generate a start instruction according to the transmission parameters.

[0151] Optionally, the local extended reality application 402 is further configured to:

[0152] Determine the target transmission mode corresponding to the operation object information carried by the viewing operation by locally querying the correspondence between the object and the data transmission mode; or,

[0153] Upload the operation object information to a cloud storage server, and receive the target transmission mode returned by the cloud storage server, wherein the cloud storage server stores the data transmission mode corresponding to each object.

[0154] Optionally, the target transmission mode is wireless streaming, the data receiving module corresponding to the wireless streaming is a wireless data receiving module, and the data sending end and the extended reality display device are in the same local area network; the stream data receiving end 404 is further configured to:

[0155] Listen to the wireless channel between the extended reality display device and the data sending end through the wireless data receiving module;

[0156] Receive the stream data transmitted by the data sending end through the wireless channel.

[0157] Optionally, the target transmission mode is wired streaming, and the data receiving module corresponding to the wired streaming is a wired data receiving module; the stream data receiving end 404 is further configured to:

[0158] determining whether a connection line exists between the extended reality display device and the data sending end;

[0159] In the case where the connection line exists, the wired data receiving module is used to monitor the connection line between the extended reality display device and the data sending end, and receive the stream data transmitted through the connection line.

[0160] Optionally, the target transmission mode is cloud rendering streaming, and the data receiving module corresponding to the cloud rendering streaming is a cloud rendering module; the stream data receiving end 404 is further configured to:

[0161] The stream server address is set through the cloud rendering module, and the stream data transmitted by the cloud server corresponding to the stream server address is received.

[0162] Optionally, the stream data receiving end 404 is further configured to:

[0163] Collecting control information of the extended reality display device and / or the control device;

[0164] According to the target transmission mode, the control information is fed back to the data sending end.

[0165] Optionally, the stream data receiving end 404 is further configured to:

[0166] In the case where the stream exit instruction is detected, the receiving of the stream data is terminated;

[0167] The scheduling interface of the local extended reality application is called, and the local extended reality application is returned.

[0168] The extended reality display device provided by the embodiments of the present specification is provided with a newly added stream data receiving end, the stream data receiving end is configured with a data receiving module corresponding to at least one data transmission mode, that is, the stream data receiving end is a data receiving end common to at least one data transmission mode, can simultaneously support the at least one data transmission mode, when a user triggers a certain viewing operation in a local extended reality application, the local extended reality application can generate a corresponding start instruction based on the viewing operation, send the start instruction to the stream data receiving end, and the stream data receiving end can determine the target transmission mode of the required stream data from at least one data transmission mode in response to the start instruction, and then start the transmission process corresponding to the target transmission mode, to realize stream data receiving and rendering. In this way, complex rendering programs can be implemented on other data sending ends outside the extended reality display device, and the extended reality display device performs data rendering and display through the stream mode, realizing seamless switching from the local extended reality application to the stream data receiving end, and the stream data receiving end can simultaneously support multiple transmission modes, thereby accurately and efficiently switching different transmission modes, avoiding the lag caused by switching from the local extended reality application to the stream data receiving end and the lag caused by switching different transmission modes, and greatly improving the user experience.

[0169] The above is a schematic scheme of an extended reality display device according to an embodiment of the present specification. It should be noted that the technical scheme of the extended reality display device belongs to the same concept as the technical scheme of the above-mentioned extended reality data rendering method, and the details of the technical scheme of the extended reality display device that are not described in detail can be referred to the description of the technical scheme of the above-mentioned extended reality data rendering method.

[0170] Referring to Figure 5 , Figure 5 A flowchart of another extended reality data rendering method according to an embodiment of the present specification is shown, applied to a stream data receiving end in an extended reality display device, the stream data receiving end is configured with a data receiving module corresponding to at least one data transmission mode, and the extended reality data rendering method specifically includes the following steps:

[0171] Step 502: receiving a start instruction, wherein the start instruction is generated and sent by a local extended reality application in an extended reality display device based on a detected viewing operation.

[0172] Step 504: in response to the start instruction, determining a target transmission mode from at least one data transmission mode.

[0173] Step 506: receiving stream data transmitted by a data sending end through a data receiving module corresponding to the target transmission mode according to a data receiving process corresponding to the target transmission mode, and performing rendering and display.

[0174] The extended reality data rendering method provided in this specification adds a streaming data receiver to the extended reality display device. This streaming data receiver is configured with a data receiving module corresponding to at least one data transmission method. That is, the streaming data receiver is a universal data receiver for at least one data transmission method and can support at least one data transmission method simultaneously. When a user triggers a viewing operation in a local extended reality application, the local extended reality application can generate a corresponding start command based on the viewing operation and send the start command to the streaming data receiver. In response to the start command, the streaming data receiver can determine the target transmission method of the required streaming data from at least one data transmission method, and then start the transmission process corresponding to the target transmission method to realize the reception and rendering of streaming data. In this way, complex rendering processes can be implemented on data sending ends other than the extended reality display device. The extended reality display device renders and displays data via streaming, achieving a seamless transition from the local extended reality application to the streaming data receiving end. Moreover, the streaming data receiving end can be compatible with multiple transmission methods simultaneously, thereby accurately and efficiently switching between different transmission methods. This avoids stuttering when switching from the local extended reality application to the streaming data receiving end, as well as stuttering caused by switching between different transmission methods, greatly improving the user experience.

[0175] The above Figure 5 This is an illustrative scheme of an extended reality data rendering method at a streaming data receiving end, as described in this embodiment. It should be noted that the technical solution of this extended reality data rendering method is similar to that described above. Figure 2 The technical solutions for extended reality data rendering methods shown belong to the same concept. Figure 5 For details not described in the technical solution of the extended reality data rendering method shown above, please refer to the above. Figure 2 The technical solution of the extended reality data rendering method shown is described.

[0176] See Figure 6 , Figure 6 This diagram illustrates a structural block diagram of a streaming data receiver according to an embodiment of this specification. The streaming data receiver is configured with a data receiving module corresponding to at least one data transmission method, such as... Figure 6 As shown, the streaming data receiver includes:

[0177] The receiving module 602 is configured to receive a start command, wherein the start command is generated and sent by the local extended reality application in the extended reality display device based on the detected viewing operation;

[0178] The determination module 604 is configured to determine a target transmission mode from at least one data transmission mode in response to a start command;

[0179] The display module 606 is configured to receive the streaming data transmitted by the data sender according to the data receiving process corresponding to the target transmission method, and then render and display it.

[0180] The streaming data receiver provided in this embodiment is configured with a data receiving module corresponding to at least one data transmission method. That is, the streaming data receiver is a universal data receiver for at least one data transmission method, simultaneously supporting all at least one data transmission method. When a user triggers a viewing operation in a local extended reality application, the local extended reality application can generate a corresponding startup command based on the viewing operation and send the startup command to the streaming data receiver. Responding to the startup command, the streaming data receiver can determine the target transmission method for the required streaming data from the at least one data transmission method, and then initiate the transmission process corresponding to the target transmission method, realizing the reception and rendering of streaming data. Thus, complex rendering programs can be implemented on other data sending ends besides the extended reality display device. The extended reality display device renders and displays data via streaming, achieving a seamless transition from the local extended reality application to the streaming data receiver. Furthermore, the streaming data receiver can simultaneously support multiple transmission methods, thereby accurately and efficiently switching between different transmission methods, avoiding stuttering when switching from the local extended reality application to the streaming data receiver, and stuttering caused by switching different transmission methods, greatly improving the user experience.

[0181] The above is an illustrative scheme of a streaming data receiver according to this embodiment. It should be noted that the technical solution of this streaming data receiver is similar to that described above. Figure 2 The extended reality data rendering method shown above belongs to the same concept. Details not described in detail in the technical solution of this streaming data receiver can be found in the above... Figure 2 The technical solution of the extended reality data rendering method shown is described.

[0182] See Figure 7 , Figure 7 The flowchart illustrates another extended reality data rendering method according to an embodiment of this specification, applied to a local extended reality application in an extended reality display device. The extended reality data rendering method specifically includes the following steps:

[0183] Step 702: Receive the viewing operation and generate a startup command based on the viewing operation, wherein the startup command carries transmission parameters.

[0184] Step 704: Send the start command to the streaming data receiver, wherein the streaming data receiver is configured with a data receiving module corresponding to at least one data transmission mode, and the transmission parameters carried in the start command are used by the streaming data receiver to determine the target transmission mode from at least one data transmission mode.

[0185] The extended reality data rendering method provided by the embodiment of the present specification can generate a corresponding starting instruction based on a user triggered viewing operation, the starting instruction carries a transmission parameter, the starting instruction is sent to a stream data receiving end, and the stream data receiving end can determine a target transmission mode of required stream data directly according to the transmission parameter carried in the starting instruction, and then start a transmission process corresponding to the target transmission mode to realize stream data receiving and rendering. The stream data receiving end is configured with a data receiving module corresponding to at least one data transmission mode, that is, the stream data receiving end is a data receiving end common to at least one data transmission mode, and can simultaneously support the at least one data transmission mode. In this way, complex rendering programs can be implemented on other data sending ends other than the extended reality display device, and the extended reality display device displays data by streaming, realizing seamless switching from local extended reality application to stream data receiving end, and the stream data receiving end can simultaneously support multiple transmission modes, thereby accurately and efficiently switching different transmission modes, avoiding stuttering when switching from local extended reality application to stream data receiving end, and stuttering caused by switching different transmission modes, greatly improving user experience.

[0186] The above Figure 7 The above Figure 2 The above Figure 5 The above Figure 2 The above

[0187] Referring to Figure 8 , Figure 8 A structural block diagram of a local extended reality application according to an embodiment of the present specification is shown, as shown in Figure 8 The local extended reality application includes:

[0188] The generating module 802 is configured to receive a viewing operation and generate a starting instruction according to the viewing operation, wherein the starting instruction carries a transmission parameter;

[0189] The sending module 804 is configured to send the starting instruction to the stream data receiving end, wherein the stream data receiving end is configured with a data receiving module corresponding to at least one data transmission mode, and the transmission parameter carried in the starting instruction is used to determine a target transmission mode from the at least one data transmission mode by the stream data receiving end.

[0190] The local extended reality application provided by the embodiment of the present specification can generate a corresponding starting instruction based on a user triggered viewing operation, the starting instruction carries a transmission parameter, the starting instruction is sent to a stream data receiving end, and the stream data receiving end can determine a target transmission mode of required stream data directly according to the transmission parameter carried in the starting instruction, and then start a transmission process corresponding to the target transmission mode, so as to realize stream data receiving and rendering. The stream data receiving end is configured with a data receiving module corresponding to at least one data transmission mode, that is, the stream data receiving end is a data receiving end common to at least one data transmission mode, and can simultaneously support the at least one data transmission mode. In this way, complex rendering programs can be implemented on other data sending ends other than the extended reality display device, and the extended reality display device performs data rendering and display through streaming, realizing seamless switching from the local extended reality application to the stream data receiving end, and the stream data receiving end can simultaneously support multiple transmission modes, thereby accurately and efficiently switching different transmission modes, avoiding stuttering when switching from the local extended reality application to the stream data receiving end, and stuttering caused by switching different transmission modes, and greatly improving user experience.

[0191] The above is a schematic scheme of a local extended reality application of the present embodiment. It should be noted that the technical scheme of the local extended reality application belongs to the same concept as the technical scheme of the extended reality data rendering method shown in the above Figure 2 The technical scheme of the local extended reality application has not been described in detail, and the details can be referred to the description of the technical scheme of the extended reality data rendering method shown in the above Figure 2 .

[0192] Referring to Figure 9 , Figure 9 a structural block diagram of an extended reality data rendering system according to an embodiment of the present specification is shown, the system comprises: a data sending end 902 and an extended reality display device 904, the data sending end 902 is provided with a data sending module 9022 corresponding to at least one data transmission mode, the extended reality display device 904 is provided with a local extended reality application 9042 and a stream data receiving end 9044, the stream data receiving end 9044 is configured with a data receiving module corresponding to at least one data transmission mode;

[0193] The data sending end 9022 is configured to collect to-be-rendered data, generate stream data according to the to-be-rendered data, and transmit the stream data to the stream data receiving end 9044 based on a corresponding target transmission mode;

[0194] The local extended reality application 9042 is configured to receive a viewing operation and generate a starting instruction according to the viewing operation, and send the starting instruction to the stream data receiving end 9044.

[0195] The stream data receiving end 9044 is configured to determine a target transmission mode from at least one data transmission mode in response to the starting instruction; receive the stream data transmitted by the data sending end 9022 through the data receiving module corresponding to the target transmission mode according to the data receiving process corresponding to the target transmission mode, and perform rendering display.

[0196] In an optional implementation of the embodiment, the viewing operation carries operation object information; the local extended reality application 9042 is further configured to query the target transmission mode corresponding to the operation object information carried by the viewing operation; determine the transmission parameter representing the target transmission mode, and generate the starting instruction according to the transmission parameter.

[0197] The stream data receiving end 9044 is further configured to determine the corresponding target transmission mode according to the transmission parameter carried in the starting instruction in the case that the starting instruction carries the transmission parameter.

[0198] In an optional implementation of the embodiment, the stream data receiving end 9044 is configured with a wireless data receiving module, the data sending end 902 and the extended reality display device 904 are in the same local area network, and the data sending end 902 and the extended reality display device 904 communicate through a wireless transmission protocol;

[0199] The data sending end 902 is configured to transmit the stream data to the wireless data receiving module along a wireless channel through the wireless transmission protocol;

[0200] The wireless data receiving module is configured to listen to the wireless channel between the extended reality display device 904 and the data sending end 902, and receive the stream data transmitted by the data sending end 902 through the wireless channel.

[0201] In an optional implementation of the embodiment, the stream data receiving end 9044 is configured with a wired data receiving module; the data sending end 902 and the extended reality display device 904 are connected through a connection line, and the data sending end 902 and the extended reality display device 904 communicate through a display port;

[0202] The data sending end 903 is configured to transmit the stream data to the corresponding display port of the extended reality display device 904 through the connection line;

[0203] The wired data receiving module is configured to listen to the connection line and obtain the stream data transmitted to the display port through the connection line.

[0204] In an optional implementation of the embodiment, the stream data receiving end 9044 is configured with a cloud rendering module, and the data sending end 902 is a cloud server;

[0205] The stream data receiving end 9044 is configured to package the cloud rendering module into an executable file, run on a cloud server, and set a stream server address and other authentication information in the cloud rendering module. After the setting is completed, the stream data transmitted by the cloud server is received through the cloud rendering module. The cloud server is the cloud server corresponding to the stream server address.

[0206] The extended reality data rendering system provided by the embodiments of the present specification adds a stream data receiving end in the extended reality display device. The stream data receiving end is configured with a data receiving module corresponding to at least one data transmission mode, i.e., the stream data receiving end is a data receiving end common to at least one data transmission mode, which can support at least one data transmission mode at the same time. When a user triggers a certain viewing operation in the local extended reality application, the local extended reality application can generate a corresponding start instruction based on the viewing operation, send the start instruction to the stream data receiving end, and the stream data receiving end can determine the target transmission mode of the required stream data from at least one data transmission mode in response to the start instruction, and then start the transmission process corresponding to the target transmission mode to realize stream data receiving and rendering. In this way, complex rendering programs can be implemented on other data sending ends outside the extended reality display device, and the extended reality display device can perform data rendering and display through the stream mode, realizing seamless switching from the local extended reality application to the stream data receiving end. In addition, the stream data receiving end can be compatible with multiple transmission modes at the same time, so as to accurately and efficiently switch different transmission modes, avoid the lag caused by switching from the local extended reality application to the stream data receiving end, and the lag caused by switching different transmission modes, and greatly improve the user experience.

[0207] The above is a schematic scheme of an extended reality data rendering system of the present embodiment. It should be noted that the technical scheme of the extended reality data rendering system belongs to the same concept as the technical scheme of the extended reality data rendering method shown in the above Figure 2 The technical scheme of the extended reality data rendering system is not described in detail, and can be referred to the description of the technical scheme of the extended reality data rendering method shown in the above Figure 2 .

[0208] The following will be described in conjunction with the accompanying Figure 10 The application of the extended reality data rendering method provided by the present specification in the e-commerce scene will be further described. Among them, Figure 10 Fig. 1 shows a processing process flowchart of a product data rendering method provided by an embodiment of the present specification, applied to a head-mounted display, which includes a local product application and a product stream data receiving end. The product stream data receiving end is configured with a data receiving module corresponding to at least one data transmission mode. The specific steps of the product data rendering method are as follows:

[0209] Step 1002: The local commodity application receives a commodity viewing operation and generates a starting instruction according to the commodity viewing operation, and sends the starting instruction to the commodity stream data receiving end.

[0210] It should be noted that the local commodity application includes many interactive scenes such as 3D stores, 3D street views, virtual live rooms, and clothing showrooms. The user can select the content he wants to watch and trigger the viewing operation. At this time, the local commodity application can generate a corresponding starting instruction based on the viewing operation and send the starting instruction to the commodity stream data receiving end.

[0211] For example, the user is wandering in the local commodity application and sees a clothing show. After the user clicks on the 3D model of the clothing show, the local commodity application can query the target transmission mode corresponding to the clothing show in the local or cloud storage server, and then generate a corresponding starting instruction based on the transmission parameters corresponding to the target transmission mode, and send the starting instruction to the commodity stream data receiving end.

[0212] Step 1004: The commodity stream data receiving end, in response to the starting instruction, determines the target transmission mode from at least one data transmission mode, receives the commodity stream data transmitted by the data sending end through the data receiving module corresponding to the target transmission mode according to the data receiving process corresponding to the target transmission mode, and performs rendering display.

[0213] The commodity data rendering method provided by the embodiments of the present specification adds a commodity stream data receiving end in the head-mounted display. The commodity stream data receiving end is configured with a data receiving module corresponding to at least one data transmission mode, i.e., the commodity stream data receiving end is a data receiving end common to at least one data transmission mode and can support the at least one data transmission mode at the same time. When the user triggers a certain viewing operation in the local commodity application, the local commodity application can generate a corresponding starting instruction based on the viewing operation and send the starting instruction to the commodity stream data receiving end. The commodity stream data receiving end responds to the starting instruction to determine the target transmission mode of the required commodity stream data from at least one data transmission mode, and then starts the transmission process corresponding to the target transmission mode to realize commodity stream data receiving and rendering. In this way, complex rendering programs can be implemented in other data sending ends other than the head-mounted display. The head-mounted display performs data rendering and display through streaming, realizes seamless switching from the local commodity application to the commodity stream data receiving end, and the commodity stream data receiving end can be compatible with multiple transmission modes at the same time, thereby accurately and efficiently switching different transmission modes, avoiding the lag from switching from the local commodity application to the commodity stream data receiving end and the lag caused by switching different transmission modes, and greatly improving the user's experience in browsing commodities in the head-mounted display.

[0214] Figure 11A structural block diagram of a computing device according to one embodiment of the present specification is shown. The components of the computing device 1100 include, but are not limited to, a memory 1110 and a processor 1120. The processor 1120 is connected with the memory 1110 through a bus 1130, and a database 1150 is used to save data.

[0215] The computing device 1100 also includes an access device 1140 that enables the computing device 1100 to communicate via one or more networks 1160. Examples of these networks include the public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1140 can include one or more of any type of network interface (e.g., network interface card (NIC)), wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a near-field communication (NFC) interface, and the like.

[0216] In one embodiment of the present specification, the above-mentioned components of the computing device 1100 and other components not shown in the above-mentioned components can be connected with each other, for example, through a bus. It should be understood that, Figure 11 the above-mentioned components of the computing device 1100 and other components not shown in the above-mentioned components can be connected with each other, for example, through a bus. It should be understood that, Figure 11 The structural block diagram of the computing device shown is only for the purpose of example, and is not a limitation on the scope of the present specification. Other components can be added or replaced as needed by those skilled in the art.

[0217] The computing device 1100 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or a PC. The computing device 1100 can also be a mobile or stationary server.

[0218] The processor 1120 is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned extended reality data rendering method.

[0219] The above is a schematic scheme of a computing device according to one embodiment of the present specification. It should be noted that the technical scheme of the computing device belongs to the same concept as the technical scheme of the above-mentioned extended reality data rendering method, and the details of the technical scheme of the computing device that are not described in detail can be referred to the description of the technical scheme of the above-mentioned extended reality data rendering method.

[0220] An embodiment of the present specification further provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions realize the steps of the above-mentioned extended reality data rendering method when executed by a processor.

[0221] The above is a schematic scheme of the computer readable storage medium of the embodiment of the present specification. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned extended reality data rendering method belong to the same concept, and the details of the technical scheme of the storage medium which are not described in detail can be referred to the description of the technical scheme of the above-mentioned extended reality data rendering method.

[0222] An embodiment of the present specification further provides a computer program, wherein the computer program causes a computer to execute the steps of the above-mentioned extended reality data rendering method when the computer program is executed in the computer.

[0223] The above is a schematic scheme of the computer program of the embodiment of the present specification. It should be noted that the technical scheme of the computer program and the technical scheme of the above-mentioned extended reality data rendering method belong to the same concept, and the details of the technical scheme of the computer program which are not described in detail can be referred to the description of the technical scheme of the above-mentioned extended reality data rendering method.

[0224] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than those in the embodiments and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order in order to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous.

[0225] The computer instructions include computer program code, which can be in the form of source code, object code, executable code, or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content contained in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0226] It should be noted that, for the aforementioned method embodiments, the sequences of the described actions are not necessarily required to implement the present application, and certain actions can be performed in other sequences, or even at the same time, in accordance with the present application. Furthermore, certain actions can not be required to implement the present application. Additionally, the described embodiments are not necessarily the only possible implementation of the present application.

[0227] In the above embodiments, the description of each embodiment is focused on the aspects of the embodiment, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0228] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The alternative embodiments do not describe all the details and limit the present application to the specific embodiments described. Obviously, according to the content of the present application, many modifications and changes can be made. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can well understand and use the present application. The present application is limited by the claims and their full scope and equivalents.

Claims

1. An extended reality data rendering method applied to an extended reality display device, the extended reality display device comprising a local extended reality application and a stream data receiver, the stream data receiver being configured with data receiving modules corresponding to at least two data transmission modes, the extended reality data rendering method comprising: the local extended reality application receiving a viewing operation and generating a start instruction according to the viewing operation, and sending the start instruction to the stream data receiver, wherein the start instruction comprises a transmission parameter, the transmission parameter indicating that the data receiving module adopts a target transmission mode determined according to operation object information carried in the viewing operation, and the operation object information is used to determine whether data corresponding to the viewing operation is data in the local extended reality application or stream data; the stream data receiver determining a target transmission mode from the at least two data transmission modes in response to the start instruction, receiving stream data transmitted by a data sending end through a data receiving module corresponding to the target transmission mode according to a data receiving process corresponding to the target transmission mode, and performing rendering display.

2. The extended reality data rendering method of claim 1, wherein the target transmission mode is determined from the at least two data transmission modes, comprising: in the case that the start instruction carries a transmission parameter, determining the corresponding target transmission mode according to the transmission parameter carried in the start instruction; or listening to the at least two data transmission modes to determine a target transmission mode that satisfies a transmission condition.

3. The extended reality data rendering method of claim 2, wherein the viewing operation carries operation object information, and the start instruction is generated according to the viewing operation, comprising: querying the target transmission mode corresponding to the operation object information carried in the viewing operation; determining a transmission parameter representing the target transmission mode, and generating the start instruction according to the transmission parameter.

4. The extended reality data rendering method of claim 3, wherein the target transmission mode corresponding to the operation object information carried in the viewing operation is determined by querying a correspondence between a local query object and a data transmission mode; or the operation object information is uploaded to a cloud storage server, and the target transmission mode returned by the cloud storage server is received, wherein the cloud storage server stores data transmission modes corresponding to respective objects.

5. The extended reality data rendering method of any one of claims 1-4, wherein the target transmission mode is wireless transmission, the data receiving module corresponding to the wireless transmission is a wireless data receiving module, the data sending end and the extended reality display device are in the same local area network, and the stream data transmitted by the data sending end is received through the data receiving module corresponding to the target transmission mode according to the data receiving process corresponding to the target transmission mode, comprising: listening to a wireless channel between the extended reality display device and the data sending end through the wireless data receiving module. ​ ​ ​ Receive the stream data transmitted by the data sending end through the wireless channel.

6. The extended reality data rendering method according to any one of claims 1-4, wherein the target transmission mode is wired transmission, and the wired transmission corresponds to a wired data receiving module; and the receiving, by the data receiving module corresponding to the target transmission mode, of the stream data transmitted by the data sending end according to the data receiving process corresponding to the target transmission mode comprises: determining whether a connection line exists between the extended reality display device and the data sending end; in the case where the connection line exists, listening to the connection line between the extended reality display device and the data sending end by the wired data receiving module, and receiving the stream data transmitted through the connection line.

7. The extended reality data rendering method according to any one of claims 1-4, wherein the target transmission mode is cloud transmission, and the cloud transmission corresponds to a cloud rendering module; and the receiving, by the data receiving module corresponding to the target transmission mode, of the stream data transmitted by the data sending end according to the data receiving process corresponding to the target transmission mode comprises: setting a stream server address by the cloud rendering module, and receiving the stream data transmitted by a cloud server corresponding to the stream server address.

8. The extended reality data rendering method according to any one of claims 1-4, wherein after the receiving of the stream data transmitted by the data sending end and the rendering and display, the method further comprises: collecting control information of the extended reality display device and / or a control device; feeding back the control information to the data sending end according to the target transmission mode.

9. The extended reality data rendering method according to any one of claims 1-4, wherein after the receiving of the stream data transmitted by the data sending end and the rendering and display, the method further comprises: terminating the receiving of the stream data in the case where a stream exit instruction is detected; calling a scheduling interface of the local extended reality application, and returning the local extended reality application.

10. An extended reality data rendering method applied to a stream data receiving end in an extended reality display device, wherein the stream data receiving end is configured with data receiving modules corresponding to at least two data transmission modes, and the extended reality data rendering method comprises: receiving a start instruction, wherein the start instruction is generated and sent by a local extended reality application in the extended reality display device based on a detected viewing operation, wherein the start instruction comprises a transmission parameter, the transmission parameter indicates that the data receiving module adopts a target transmission mode, and the target transmission mode is determined according to operation object information carried in the viewing operation, and the operation object information is used to determine whether data corresponding to the viewing operation is data in the local extended reality application or stream data; determining the target transmission mode from the at least two data transmission modes in response to the start instruction; receiving, by the data receiving module corresponding to the target transmission mode, stream data transmitted by a data sending end according to a data receiving process corresponding to the target transmission mode, and rendering and displaying the stream data.

11. An extended reality data rendering method applied to a local extended reality application in an extended reality display device, the method comprising: receiving a viewing operation and generating a start instruction according to the viewing operation, wherein the start instruction carries a transmission parameter indicating that a data receiving module adopts a corresponding target transmission mode, and the target transmission mode is determined according to operation object information carried in the viewing operation, and the operation object information is used to determine whether data corresponding to the viewing operation is data in the local extended reality application or streaming data; sending the start instruction to a streaming data receiving end, wherein the streaming data receiving end is configured with data receiving modules corresponding to at least two data transmission modes, and the transmission parameter carried in the start instruction is used to determine a target transmission mode from the at least two data transmission modes by the streaming data receiving end.

12. An extended reality data rendering system, the system comprising: A data sending end and an extended reality display device, the data sending end is provided with data sending modules corresponding to at least two data transmission modes, and the extended reality display device is provided with a local extended reality application and a streaming data receiving end, and the streaming data receiving end is configured with data receiving modules corresponding to at least two data transmission modes; The data sending end is configured to collect to-be-rendered data, generate streaming data according to the to-be-rendered data, and transmit the streaming data to the streaming data receiving end based on a corresponding target transmission mode; The local extended reality application is configured to receive a viewing operation and generate a start instruction according to the viewing operation, and send the start instruction to the streaming data receiving end, wherein the start instruction includes a transmission parameter, the transmission parameter indicates that the data receiving module adopts a corresponding target transmission mode, the target transmission mode is determined according to operation object information carried in the viewing operation, and the operation object information is used to determine whether data corresponding to the viewing operation is data in the local extended reality application or streaming data; The streaming data receiving end is configured to determine a target transmission mode from the at least two data transmission modes in response to the start instruction; receive and render streaming data transmitted by the data sending end through a data receiving module corresponding to the target transmission mode according to a data receiving process corresponding to the target transmission mode.

13. A commodity data rendering method applied to a head-mounted display, the head-mounted display comprising a local commodity application and a commodity streaming data receiving end, the commodity streaming data receiving end being configured with data receiving modules corresponding to at least two data transmission modes, and the commodity data rendering method comprising: The local commodity application receives a commodity viewing operation and generates starting instruction according to the commodity viewing operation, and sends the starting instruction to the commodity stream data receiving end, wherein the starting instruction comprises a transmission parameter, the transmission parameter indicates that the data receiving module adopts a corresponding target transmission mode, the target transmission mode is determined according to operation object information carried in the viewing operation, and the operation object information is used to determine whether data corresponding to the viewing operation is data in the local commodity application or stream data; The commodity stream data receiving end determines a target transmission mode from the at least two data transmission modes in response to the starting instruction, receives commodity stream data transmitted by the data sending end through a data receiving module corresponding to the target transmission mode according to a data receiving process corresponding to the target transmission mode, and performs rendering and display.

Images