Method, device, equipment, and computer program for loading a virtual scene

Abstract

The present application provides a method, apparatus, device, computer-readable storage medium, and computer program product for loading a virtual scene. The embodiments of the present application can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation, and vehicle-mounted, and are related to cloud computing technology. The method includes: receiving a stream media playback address of a first protocol that a cloud terminal device responds to an address pull request; in response to a load trigger operation for the virtual scene, sending a media screen projection request corresponding to a local media file to the playback device, and sending a load command to the cloud terminal device; in response to a parameter acquisition request that the playback device responds to the media screen projection request, obtaining initial file parameters corresponding to the local media file; using the stream media playback address to replace the local file address of a second protocol in the initial file parameters to obtain target file parameters; and returning the target file parameters to the playback device.

Description

[Technical field]

[0001] The present application relates to cloud computing technology, and in particular to a method, an apparatus, a device, a computer-readable storage medium, and a computer program product for loading a virtual scene.

[0002] This application is based on and claims priority to a Chinese patent application having application number 202210271200.5 and filing date March 18, 2022, the entire contents of which are hereby incorporated by reference into this application. [Background technology]

[0003] A cloud gaming application runs a game on a cloud terminal device and transmits the game screen and audio to a playback device via a network, making it possible to use playback devices with relatively low data processing capabilities, such as smart TVs and projectors, and also to load virtual scenes when running a cloud game.

[0004] In the related technology, a cloud game client usually needs to be installed on the playback device first, and when a player launches any cloud game, the cloud terminal device loads the virtual scene of the cloud game in the background, and the cloud game client controls the interaction between the playback device and the cloud terminal device to obtain the stream media data of the virtual scene.

[0005] However, due to differences in data transmission protocols, hardware configurations, etc., the types of playback devices on which the cloud gaming client can be installed are limited, resulting in relatively few playback devices that allow virtual scenes to be loaded, i.e., the range of devices that support the loading of virtual scenes is relatively small. Summary of the Invention [Problem to be solved by the invention]

[0006] SUMMARY OF THE DISCLOSURE The embodiments of the present application provide a method, apparatus, device, computer-readable storage medium, and computer program product for loading a virtual scene, enabling an expansion of the range of devices that support loading a virtual scene.

[0007] The technical solutions of the embodiments of the present application are realized as follows:

[0008] An embodiment of the present application provides a method for loading a virtual scene, executed by a control device, the method comprising: receiving a stream media playback address of a first protocol returned by the cloud terminal device in response to the address pull request; In response to a load trigger operation for the virtual scene, sending a media screen projection request corresponding to a local media file to a playback device, and sending a load command to instruct the cloud terminal device to start loading the virtual scene; obtaining initial file parameters corresponding to the local media file in response to a parameter obtainment request sent by the playback device in response to the media screen projection request; Using the streaming media playback address to replace the local file address of the second protocol in the initial file parameter to obtain a target file parameter, the streaming media playback address being used for providing the streaming media data of the virtual scene generated when the cloud terminal device responds to the loading command; returning the target file parameters to the playback device, the target file parameters being used to provide the stream media playback address to the playback device.

[0009] An embodiment of the present application provides a method for loading a virtual scene, executed by a playback device, the method comprising: receiving a media screen projection request for a local media file transmitted from a control device; returning a parameter acquisition request to the control device in response to the media screen projection request; analyzing and acquiring a stream media playback address of a first protocol from the target file parameters returned by the control device in response to the parameter acquisition request, the target file parameters being acquired by replacing a local file address of a second protocol in an initial file parameter with the stream media playback address, the initial file parameter corresponding to the local media file; Pulling and obtaining stream media data of a virtual scene from the stream media playback address, and playing the stream media data.

[0010] An embodiment of the present application provides a virtual scene loading method executed by a cloud terminal device, the method comprising: generating a stream media playback address of a first protocol in response to an address pull request sent by the control device; returning the stream media playback address to the control device, where the stream media playback address is used to replace a local file address of the second protocol in an initial file parameter and obtain a target file parameter, where the initial file parameter corresponds to a local media file of the control device; receiving a load command sent by the control device, the load command being used to initiate loading of the virtual scene; In response to the load command, loading the virtual scene, obtaining stream media data corresponding to the virtual scene, and storing the stream media data at the stream media playback address.

[0011] An embodiment of the present application provides a first device for loading a virtual scene; A first receiving module is configured to receive a stream media playback address of a first protocol that is returned by a cloud terminal device in response to an address pull request; A first sending module is configured to send a media screen projection request corresponding to a local media file to a playback device in response to a load trigger operation on the virtual scene, and send a load command to the cloud terminal device to instruct the cloud terminal device to start loading the virtual scene; a parameter generation module configured to obtain initial file parameters corresponding to the local media file in response to a parameter acquisition request returned by the playback device for the media screen projection request, and to use the stream media playback address to replace the local file address of a second protocol in the initial file parameters to obtain target file parameters, wherein the stream media playback address is used for providing stream media data of the virtual scene generated by the cloud terminal device when the cloud terminal device responds to the load command; The first sending module is further configured to return the target file parameters to the playback device, where the target file parameters are used to provide the stream media playback address to the playback device.

[0012] An embodiment of the present application provides a second device for loading the virtual scene; a second receiving module configured to receive a media screen projection request for the local media file transmitted from the control device; a second sending module configured to send a parameter acquisition request back to the control device in response to the media screen projection request; a data request module configured to: analyze and obtain a stream media playback address of a first protocol from a target file parameter returned by the control device in response to the parameter acquisition request, the target file parameter being obtained by replacing a local file address of a second protocol in an initial file parameter with the stream media playback address, the initial file parameter corresponding to the local media file; and pull and obtain stream media data of a virtual scene from the stream media playback address; and a data playback module configured to play the stream media data.

[0013] An embodiment of the present application provides a third device for loading the virtual scene; an address generation module configured to generate a stream media playback address of the first protocol in response to an address pull request sent by the control device; a third sending module configured to return the stream media playback address to the control device, the stream media playback address being used to replace a local file address of a second protocol in an initial file parameter and obtain a target file parameter, the initial file parameter corresponding to a local media file of the control device; a third receiving module configured to receive a load command sent by the control device, the load command being used to instruct the control device to start loading the virtual scene; and a scene load module configured to, in response to the load instruction, load the virtual scene, obtain stream media data corresponding to the virtual scene, and store the stream media data at the stream media playback address.

[0014] An embodiment of the present application provides a control device, the control device comprising: a first memory adapted to store computer executable instructions; and a first processor, which is used to realize the control device side virtual scene loading method provided by the embodiment of the present application when executing the computer-executable instructions stored in the first memory.

[0015] An embodiment of the present application provides a playback device, the playback device comprising: a second memory adapted to store computer executable instructions; and a second processor, which is used to realize the playback device-side virtual scene loading method provided by the embodiment of the present application when executing the computer-executable instructions stored in the second memory.

[0016] An embodiment of the present application provides a cloud terminal device, the cloud terminal device comprising: a third memory adapted to store computer executable instructions; and and a third processor, which is used to realize the cloud terminal device side virtual scene loading method provided by the embodiment of the present application when executing the computer-executable instructions stored in the third memory.

[0017] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a first processor, realizes a virtual scene loading method on a control device side provided by an embodiment of the present application, when executed by a second processor, realizes a virtual scene loading method on a playback device side provided by an embodiment of the present application, and when executed by a third processor, realizes a virtual scene loading method on a cloud terminal device side provided by an embodiment of the present application.

[0018] An embodiment of the present application provides a computer program product, comprising a computer program or computer-executable instructions, which, when executed by a first processor, realizes the method for loading a virtual scene on a control device side provided by an embodiment of the present application, when executed by a second processor, realizes the method for loading a virtual scene on a playback device side provided by an embodiment of the present application, and when executed by a third processor, realizes the method for loading a virtual scene on a cloud terminal device side provided by an embodiment of the present application. Effect of the Invention

[0019] The embodiment of the present application has the following beneficial effects: the control device first pulls the stream media playback address from the cloud terminal device for the virtual scene, and when it starts to load the virtual scene, it triggers the screen projection receiving processing flow of the playback device by sending a media screen projection request for the local media file to the playback device. Then, in response to the parameter acquisition request sent by the playback device, it modifies the local file address in the initial file parameter corresponding to the local media file to the stream media playback address of the first protocol, and returns the acquired target file parameter to the playback device. Thus, when the playback device plays, it pulls and plays the stream media data of the virtual scene from the stream media playback address provided by the target file parameter, thereby enabling the playback device to play the stream media data of the virtual scene even when the playback device does not support the cloud game client, realizing the loading of the virtual scene, and finally expanding the range of devices that support the loading of the virtual scene. [Brief description of the drawings]

[0020] [Figure 1] FIG. 2 is a schematic diagram of the DLNA (registered trademark) standard that a control device must implement. [Diagram 2] FIG. 1 is a schematic diagram of a cloud gaming client interface. [Diagram 3]FIG. 1 is an architecture schematic diagram of a virtual scene loading system provided by an embodiment of the present application. [Figure 4] FIG. 4 is a structural schematic diagram of the control pad in FIG. 3 provided by an embodiment of the present application; [Diagram 5] FIG. 4 is a structural schematic diagram of the smart TV in FIG. 3 provided by an embodiment of the present application; [Figure 6] FIG. 4 is a structural schematic diagram of a server in FIG. 3 according to an embodiment of the present application; [Figure 7] FIG. 2 is a schematic diagram of a flow of a virtual scene loading method provided by an embodiment of the present application; [Figure 8] FIG. 2 is a schematic diagram of target file parameter generation provided by an embodiment of the present application; [Figure 9] FIG. 2 is a schematic diagram of a playback device provided in an embodiment of the present application pulling stream media data; [Figure 10] FIG. 2 is a schematic diagram of a request receiving prompt message provided by an embodiment of the present application; [Figure 11] FIG. 13 is a schematic diagram of another flow of a virtual scene loading method provided by an embodiment of the present application; [Figure 12] FIG. 2 is a schematic diagram of a failure prompt message provided by an embodiment of the present application; [Figure 13] 1 is a schematic diagram showing the appearance of a control pad provided by the present application; [Figure 14] FIG. 2 is an operational schematic diagram of a cloud game provided by an embodiment of the present application. [Figure 15] 2 is a schematic diagram of a process of encapsulating a game list provided by an embodiment of the present application into a media object in a DLNA media content service; [Figure 16] FIG. 2 is a schematic diagram illustrating a preset protocol standard playback address provided by an embodiment of the present application being replaced with a stream media playback address provided by a cloud terminal device; [Figure 17] FIG. 2 is a schematic diagram of the DLNA standard that the gamepad provided in the embodiment of the present application must implement. [Figure 18] FIG. 2 is a schematic diagram of the operation process of a cloud game provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is described in more detail below in conjunction with the drawings. The described embodiments should not be considered as limiting the present application, and all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0022] The following description refers to "some embodiments," which describe a subset of all possible embodiments, but it will be understood that "some embodiments" may be the same or different subsets of all possible embodiments, and may be combined with each other in non-inconsistent circumstances.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to be limiting of this application.

[0024] Before describing the embodiments of the present application in more detail, the relevant nouns and terms in the embodiments of the present application will be explained. The relevant nouns and terms in the embodiments of the present application are applied to the following interpretation.

[0025] 1) Cloud Computing refers to the delivery and use of IT infrastructure to obtain the required resources on-demand and in a scalable manner over a network. In a broad sense, Cloud Computing refers to the delivery and use of services to obtain the required services on-demand and in a scalable manner over a network. Such services may be related to IT, software, and the Internet, or may be other services. Cloud Computing is a product of the fusion of developments in traditional computer and network technologies such as Grid Computing, Distributed Computing, Parallel Computing, Utility Computing, Network Storage Technologies, Virtualization, and Load Balance.

[0026] With the development of the Internet, real-time data streams, and the diversification of connected devices, as well as the promotion of needs for search services, social networks, mobile business, and open collaboration, cloud computing is developing rapidly. Different from the previous parallel and distributed computing, the emergence of cloud computing is promoting a revolutionary change in the entire Internet mode and enterprise management mode from the perspective of ideas.

[0027] 2) Cloud Gaming, also known as Gaming on Demand, is a type of online gaming technology based on cloud computing technology. Cloud gaming technology enables high-quality games to be run on thin client devices with relatively limited graphics processing and data computing capabilities. In a cloud gaming scenario, a game does not run on a player's game terminal, but on a server (e.g., a cloud terminal device), and the server renders the game's virtual scene into a video and audio stream and transmits it to the game terminal via a network. The game terminal does not need to have high graphics computing capabilities and data processing capabilities, but only needs to have basic streaming media playback capabilities and the ability to obtain the player's input commands and send them to the server. The playback device in the embodiment of the present application can be regarded as a game terminal.

[0028] 3) The control device is a device that controls the launch and operation of the cloud game, such as a gamepad or a remote control.

[0029] 4) Universal Plug and Play (UPnP) protocol is a type of interconnection and communication protocol between devices, which provides functions including addressing and mutual discovery of devices.

[0030] 5) Digital Living Network Alliance (DLNA) standard is a standard established based on UPnP for devices to share content services, and different devices can access each other's services. As should be explained, DLNA is a kind of solution scheme, and includes multiple protocols, such as Hyper Text Transfer Protocol (HTTP) and Hyper Text Transfer Protocol over Secure Socket Layer (HTTPS).

[0031] 6) Media Server provides media content services according to DLNA standards. The Media Server can be applied in a storage device, thereby facilitating access to contents such as pictures and videos in the storage device.

[0032] The media server is used to implement standard interfaces such as a content directory service, a connection manager service, and an AV transport service standard.

[0033] Here, the content directory service is used to provide query and index services for media content (e.g., pictures, audio, and video, etc.), the connection management service is used to manage connections between media servers and media players, and the media transmission service standard is used to provide interfaces such as media playback address, start playback, and pause playback.

[0034] 7) Media Player (Media Render) provides media display services according to DLNA standards. The media player can be applied to playback devices such as smart TVs, projectors, and car-mounted terminals. That is, the playback devices in the embodiments of the present application support the DLNA standards.

[0035] The media player is used to implement standard interfaces such as a rendering control interface (Rendering Control), a connection management service (Connection Manager), and an AV Transport standard.

[0036] Here, the presentation control interface is used to provide an interface for setting playback attributes (e.g., brightness, contrast, and volume, etc.), and the use of the connection management service and media transmission service standards is as already described in the media server section.

[0037] 8) Regarding the Control Point, a controller implemented according to the DLNA standard is used to control the display of content by coordinating a media server and a media player. For example, a mobile terminal can implement the Control Point standard, so that a video APP on the mobile terminal can project a video onto a screen and play it on a playback device such as a television. The control device in the embodiment of the present application can be understood as a Control Point.

[0038] As should be explained, the control point coordinates between the media server and the media player by invoking standard interfaces implemented by the media server and by invoking standard interfaces implemented by the media player.

[0039] 1 is a schematic diagram of the DLNA standard that a control device needs to realize. The control device needs to realize a control point 1-1 and coordinate the operations of a media server 1-2 and a media player 1-3 through the control point 1-1. Here, the control point 1-1 includes an interface application service 1-11, which is used to present useful information to a user. The media server 1-2 includes a content directory service 1-21, a connection management service 1-22, and a media transmission service standard 1-23, and the media player 1-3 includes a program control service 1-31, a connection management service 1-32, and a media transmission service standard 1-33, and the control point 1-1 coordinates the operations of the media server 1-2 and the media player 1-3. In practice, the content directory service 1-21, the connection management service 1-22, and the media transmission service standard 1-23 cooperate with the program control service 1-31, the connection management service 1-32, and the media transmission service standard 1-33 in their operations.

[0040] 9) Real Time Streaming Protocol (RTSP) is a protocol that encodes and transmits audio and video data in real time.

[0041] 10) "Responsive to" is used to express a condition or state upon which an operation to be performed depends. The operation or operations to be performed upon satisfaction of the dependent condition or state may be in real time or may have a set delay, and unless otherwise specified, there is no restriction on the order of the operations to be performed.

[0042] 11) A virtual scene is a virtual scene displayed (or provided) on a display device. The virtual scene may be a simulation environment for the real world, a semi-simulated semi-virtual virtual environment, or even a purely virtual virtual environment. The virtual scene may be any one of a 2D virtual scene, a 2.5D virtual scene, or a 3D virtual scene. The virtual scene in the embodiment of the present application may be a virtual scene in a game application or a virtual reality (VR) application.

[0043] 12) A virtual object is an image of various people and things that can be interacted with in a virtual scene, or an actionable object in a virtual scene. The actionable object can be a virtual person, a virtual animal, an animated person, etc., such as a person, an animal, a plant, a wall, and a stone displayed in a virtual scene. The virtual object can be a virtual image used to represent a virtual user in the virtual scene. A virtual scene can include multiple virtual objects, and each virtual object has its own shape and volume in the virtual scene and occupies a part of the space in the virtual scene.

[0044] A cloud gaming application runs a game on a server and transmits the game screen and audio to the playback device via a network, making it possible to use playback devices with relatively low data processing capabilities, such as smart TVs, projectors, and in-car devices, and also to load virtual scenes when running a cloud game.

[0045] In related technologies, a cloud game client typically needs to be installed on a playback device first, so that when a player (i.e., a user) launches any cloud game, the cloud game client instructs the server to load a virtual scene of the cloud game in the background, and the cloud game client controls the playback device to interact with the server to obtain screens and audio of the virtual scene, and controls the playback device to present the obtained screens and audio.

[0046] As should be explained, the cloud game client is different from a normal game client in that the game client runs the corresponding game locally, but the cloud game client corresponds to a game selection platform, so that the user can select different cloud games and call the server to load the cloud game selected by the user.

[0047] Illustratively, FIG. 2 is a schematic diagram of a cloud game client interface. As shown in FIG. 2, the cloud game client interface 2-1 includes two areas, a control option area 2-11 and a game selection area 2-12. In the control option area 2-11, basic information 2-111 of the player (e.g., the player's nickname and rank, etc.), a display control option card 2-112, and an operation option card 2-113 (e.g., option cards such as system settings and notification prompts, etc.) are presented. In the game selection area 2-12, a game 2-121 that the player can select to launch is presented, so that the player selects the game that needs to be launched.

[0048] However, due to differences in data transmission protocols and hardware configurations, the types of playback devices on which the cloud gaming client can be installed are limited. For example, some brands of smart TVs cannot install the cloud gaming client. Therefore, the number of playback devices that allow loading virtual scenes is relatively small, that is, the range of devices that support loading virtual scenes is relatively small.

[0049] The embodiments of the present application provide a method, an apparatus, an apparatus, a computer-readable storage medium, and a computer program product for loading a virtual scene, enabling the range of apparatuses supporting loading a virtual scene to be expanded. Hereinafter, exemplary applications of the control apparatus, the playback apparatus, and the cloud terminal apparatus provided by the embodiments of the present application will be described. The playback apparatus provided by the embodiments of the present application may be implemented as various types of terminals such as a notebook computer, a tablet computer, a desktop computer, a smart TV, and an in-vehicle terminal, the control apparatus may be implemented as a control pad, a sensory apparatus, and the like, and the cloud terminal apparatus may be implemented as a server. Hereinafter, exemplary applications when the control apparatus is implemented as a smart TV, the control apparatus is implemented as a control pad, and the cloud terminal apparatus is implemented as a server will be described.

[0050] Referring to FIG. 3, FIG. 3 is an architecture schematic diagram of a virtual scene loading system provided by an embodiment of the present application. In order to realize supporting one virtual scene loading application, in the virtual scene loading system 100, a control pad 500 (control device) and a smart TV 400 (playback device) are connected to a server 200 (cloud terminal device) by a network 300, and the network 300 may be a wide area network, or a local area network, and / or a combination of both. In the virtual scene loading system 100, a database 600 is further installed, which is used to provide data support for the server 200. The database 600 may be installed in the server 200, or may be independent from the server 200. What is shown in FIG. 3 is a situation in which the database 600 is independent from the server 200.

[0051] The control pad 500 is used for receiving a stream media playback address of the first protocol returned by the server 200 in response to the address pull request; in response to a load trigger operation for the virtual scene, sending a media screen projection request corresponding to the local media file to the smart TV 400 and sending a load command to instruct the server 200 to start loading the virtual scene; in response to a parameter acquisition request returned by the smart TV 400 in response to the media screen projection request, obtaining initial media parameters corresponding to the local media file; using the stream media playback address to replace the local file address of the second protocol in the initial file parameters to obtain target file parameters; and returning the target file parameters to the smart TV 400.

[0052] The smart TV 400 is used for receiving a media screen projection request sent by the control pad 500 for a local media file, and in response to the media screen projection request, returning a parameter obtainment request to the control pad 500, parsing and obtaining a stream media playback address of the first protocol from the target file parameters returned by the control pad 500 in response to the parameter obtainment request, pulling and obtaining stream media data of the virtual scene from the stream media playback address, and playing the stream media data.

[0053] The server 200 is used for generating a stream media playback address of the first protocol in response to an address pull request sent by the control pad 500, returning the stream media playback address to the control pad 500, receiving a load command sent by the control pad 500, used to instruct the control pad 500 to start loading a virtual scene, and in response to the load command, loading the virtual scene, obtaining stream media data corresponding to the virtual scene, and storing the stream media data at the stream media playback address.

[0054] In some embodiments, the server 200 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, secure services, CDN, and big data and artificial intelligence platforms. The server 200, the smart TV 400, and the control pad 500 may be directly or indirectly connected by wired or wireless communication methods, and the embodiments of the present application are not limited thereto.

[0055] Referring to FIG. 4, FIG. 4 is a structural schematic diagram of the control pad (one embodiment of the control device) in FIG. 3 provided by an embodiment of the present application. The control pad 500 shown in FIG. 4 includes at least one first processor 510, a first memory 550, at least one first network interface 520, and a first user interface 530. Each component in the control pad 500 is coupled together by a first bus system 540. As can be understood, the first bus system 540 is used to realize the connection communication between these components. In addition to including a data bus, the first bus system 540 further includes a power bus, a control bus, and a status signal bus. For the sake of clarity, all the various buses are represented as the first bus system 540 in FIG. 4.

[0056] The first processor 510 may be a kind of integrated circuit chip, and has a signal processing capability, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Here, the general-purpose processor may be a microprocessor, or any ordinary processor, etc.

[0057] The first user interface 530 includes one or more first output devices 531 that enable media content to be presented, including one or more speakers and / or one or more visual display screens. The first user interface 530 further includes one or more first input devices 532, including user interface members that contribute to user input, such as a keyboard, a mouse, a microphone, a touch-screen display screen, a camera, other input buttons, and controls.

[0058] The first memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, etc. The first memory 550 includes one or more storage devices that are selectively remote in physical location from the first processor 510.

[0059] The first memory 550 may include volatile memory, or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read only memory (ROM), and the volatile memory may be random access memory (RAM). The first memory 550 described in the embodiments of the present application is intended to include any suitable type of memory.

[0060] In some embodiments, the first memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as illustratively described below.

[0061] The first operating system 551 is a system program used to process various basic system services and execute hardware-related tasks, such as a framework layer, a core library layer, and a driver layer, and is used to realize various basic operations and process hardware-based tasks; The first network communication module 552 is used to reach other computing devices via one or more (wired or wireless) first network interfaces 520, example first network interfaces 520 include Bluetooth, Wireless Compatibility (WiFi), Universal Serial Bus (USB), etc. a first presentation module 553 that is used to enable presentation of information (e.g., a user interface used to operate peripherals and display content and information) via one or more first output devices 531 (e.g., a display screen, a speaker, etc.) associated with the first user interface 530; The first input processing module 554 is used to detect one or more user inputs or interactivity from one of the one or more first input devices 532 and to interpret the detected inputs or interactivity.

[0062] In some embodiments, the first device that loads the virtual scene provided by the embodiments of the present application can be realized by adopting a software method. In FIG. 4, a first device 555 is shown stored in a first memory 550. It may be software in the form of a program, plug-in, etc., and includes the following software modules: a first receiving module 5551, a first transmitting module 5552, a parameter generating module 5553, a data converting module 5554, an information selecting module 555, and a display control module 5556. These modules are logical, and therefore can be arbitrarily combined or further divided according to the realized functions. The following describes the functions of each module.

[0063] Referring to FIG. 5, FIG. 5 is a structural schematic diagram of a smart TV (one embodiment of a playback device) in FIG. 3 provided by an embodiment of the present application. The smart TV 400 shown in FIG. 5 includes at least one second processor 410, a second memory 450, at least one second network interface 420, and a second user interface 430. Each component in the smart TV 400 is coupled together by a second bus system 440. As can be understood, the second bus system 440 is used to realize the connection communication between these components. In addition to including a data bus, the second bus system 440 further includes a power bus, a control bus, and a status signal bus. For the sake of clarity, all buses in FIG. 5 are represented as the second bus system 440.

[0064] The second processor 410 may be a kind of integrated circuit chip, and has a signal processing capability, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Here, the general-purpose processor may be a microprocessor, or any ordinary processor, etc.

[0065] The implementation of the second user interface 430 is similar to that of the first user interface 530 and will not be described again.

[0066] In some embodiments, the second memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as illustratively described below.

[0067] The implementations of the second operating system 451, the second network communication module 452, the second presentation module 453, and the second input processing module 454 are similar to those of the first operating system 551, the first network communication module 552, the first presentation module 553, and the first input processing module 554, respectively, and will not be repeated.

[0068] In some embodiments, the second device that loads the virtual scene provided by the embodiments of the present application can be realized by adopting a software method. In FIG. 5, a second device 455 stored in a second memory 450 is shown. It may be software in the form of a program, plug-in, etc., and includes the following software modules: a second receiving module 4551, a second sending module 4552, a data request module 4553, and a data playing module 4554. These modules are logical, and therefore can be arbitrarily combined or further divided according to the realized functions. The following describes the functions of each module.

[0069] Referring to FIG. 6, FIG. 6 is a structural schematic diagram of a server (one embodiment of a cloud terminal device) in FIG. 3 provided by an embodiment of the present application. The server 200 shown in FIG. 6 includes at least one third processor 210, a third memory 250, at least one third network interface 220, and a third user interface 230. Each component in the server 200 is coupled together by a third bus system 240. As can be understood, the third bus system 240 is used to realize the connection communication between these components. In addition to including a data bus, the third bus system 240 further includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, all buses in FIG. 6 are represented as the third bus system 240.

[0070] The third processor 210 may be a kind of integrated circuit chip, and has a signal processing capability, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Here, the general-purpose processor may be a microprocessor, or any ordinary processor, etc.

[0071] The implementation of the third user interface 230 is similar to that of the first user interface 530 and will not be described again.

[0072] In some embodiments, the third memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as illustratively described below.

[0073] The implementations of the third operating system 251, the third network communication module 252, the third presentation module 253, and the third input module are similar to those of the first operating system 551, the first network communication module 552, the first presentation module 553, and the first input processing module 554, respectively, and will not be repeated.

[0074] In some embodiments, the third device for loading the virtual scene provided by the embodiments of the present application can be realized by adopting a software method. In FIG. 6, a third device 255 is shown stored in a third memory 250. It may be software in the form of a program, plug-in, etc., and includes the following software modules: an address generating module 2551, a third sending module 2552, a third receiving module 2553, and a scene loading module 2554. These modules are logical, and therefore can be arbitrarily combined or further divided according to the realized functions. The following describes the functions of each module.

[0075] In some embodiments, the control pad (control device), smart TV (playback device), and server (cloud terminal device) can realize the virtual scene loading method provided by the embodiments of the present application by running a computer program. For example, the computer program may be a native program or software module in an operating system, a local (Native) application program (APP, Application), i.e., a program that needs to be installed in an operating system to operate, such as a cloud game connection assistant APP, or a mini program, i.e., a program that can be operated simply by downloading in a browser environment, and may even be a mini program that allows embedding in any APP. In short, the computer program may be any form of application program, module, or plug-in.

[0076] The embodiments of the present application can be applied to various virtual scene loading scenes such as cloud technology, artificial intelligence, intelligent traffic, and vehicle-mounted, etc. Hereinafter, the virtual scene loading method provided by the embodiments of the present application will be described in conjunction with the exemplary application and implementation of the control device, playback device, and cloud terminal device provided by the embodiments of the present application.

[0077] Please refer to FIG. 7, which is a schematic diagram of one flow of a method for loading a virtual scene provided by an embodiment of the present application, which will be described in conjunction with the steps shown in FIG.

[0078] S101: The cloud terminal device generates a stream media playback address of a first protocol in response to an address pull request sent by a control device.

[0079] The embodiment of the present application is realized in a scene of loading a virtual scene. For example, by loading a virtual scene corresponding to a cloud game, a user can have an interactive experience with the virtual scene. When starting to load the virtual scene, the control device first pulls the playback address of the stream media data corresponding to the virtual scene from the cloud terminal device, and then provides the playback address to the playback device, so as to facilitate the playback device to obtain and play the stream media data from the playback address. Therefore, in the embodiment of the present application, the control device needs to first generate an address pull request for the virtual scene, and send the address pull request to the cloud terminal device through the network. After receiving the address pull request, the cloud terminal device will generate a stream media playback address according to the first protocol, which is used to provide the stream media data generated when loading the virtual scene. That is, in the embodiment of the present application, the address pull request is used to pull the playback address for providing the stream media data of the virtual scene.

[0080] As should be explained, the first protocol may be a Real Time Streaming Protocol (RSTP) or a Real Time Messaging Protocol (RTMP), so that the format of the stream media playback address may be rtsp: / / xxxxxx or rtmp: / / xxxxx.

[0081] As can be appreciated, the streaming media data may include video views of the virtual scene, e.g., the appearance of the surrounding scene changes as the virtual object walks around, and may further include audio of the virtual scene, e.g., water flow sound effects in the virtual scene, etc.

[0082] It can be further understood that the virtual scene may be a virtual scene in a cloud gaming application, or a virtual scene in a virtual reality application, and the embodiment of the present application is not limited thereto. The address pull request is equipped with an identifier of the application to which the virtual scene belongs, and the cloud terminal device analyzes and obtains the identifier from the address pull request, and determines the virtual scene that needs to generate a corresponding stream media playback address according to the identifier.

[0083] In an embodiment of the present application, a wireless connection module, for example a Wi-Fi module, can be installed in the control device, so that the control device can communicate with the cloud terminal device by joining a network through the wireless connection module.

[0084] S102: The cloud terminal device returns the stream media playback address to the control device.

[0085] After generating the stream media playback address, the cloud terminal device sends the stream media playback address to the control device via the network, thereby completing the response to the address pull request. The control device receives the stream media playback address of the first protocol that the cloud terminal device responds to the address pull request. That is, the stream media playback address is the response that the cloud terminal device sends to the address pull request.

[0086] As is necessary to explain, S101 and S102 may be executed and completed before receiving a load trigger operation of the virtual scene. That is, the control device first obtains a stream media playback address from the cloud terminal device, and then directly uses it when receiving a load trigger operation of the virtual scene. At this time, the address pull request in S101 may be generated and sent by the control device in response to an add operation of the user, and the add operation refers to an operation of adding an application corresponding to the virtual scene to an application library corresponding to the user. For example, when a user adds a game to "my game", the address pull request is generated and sent to the cloud terminal device. S101 and S102 may be executed in response to a load trigger operation of the virtual scene. That is, when the control device receives a load trigger operation of the user's virtual scene, the control device may synchronously perform the process of pulling the stream media playback address from the cloud terminal device and sending the load command, and the process is not specifically limited here. In the embodiment of the present application, the process of first executing S101-S102 and then starting to respond to the load trigger operation of the virtual scene is taken as an example to describe the load method of the virtual scene in the embodiment of the present application.

[0087] S103: In response to a load trigger operation for the virtual scene, the control device sends a media screen projection request corresponding to the local media file to the playback device, and sends a load command to instruct the cloud terminal device to start loading the virtual scene.

[0088] The control device detects in real time whether a user performs a load trigger operation on the virtual scene, and when the load trigger operation is detected, performs file scanning locally to obtain a local media file. The control device also generates a media screen projection request for the local media file and sends it to the playback device, and calls the playback device through the media screen projection request to perform screen projection. The playback device receives the media screen projection request for the local media file sent from the control device. At the same time, the control device can also generate a load command, which is used to instruct the cloud terminal device to start loading the virtual scene, and send the load command to the cloud terminal device via the network. The cloud terminal device receives the load command sent by the control device, which is used to instruct the cloud terminal device to start loading the virtual scene.

[0089] As can be understood, the detection of the load trigger operation can be completed through whether or not the physical press key on the control device is operated by the user. For example, when the user presses the start key on the control pad (one implementation of the control device), the control pad determines that the load trigger operation is detected. The detection of the load trigger operation can also be completed through whether or not the control device is moved. For example, when the user shakes the haptic device (another implementation of the control device) and reaches an angle threshold, etc., the haptic device determines that the load trigger operation is detected. The embodiments of the present application are not specifically limited here.

[0090] The local media file may be a historical media file stored in the control device. For example, when the cloud terminal device receives a stop command (used to instruct the cloud terminal device to stop loading the virtual scene) for the virtual scene of the control device in the historical time, the stream media data of the virtual scene 1 s before receiving the stop command is saved, and the historical media file is obtained and sent to the control device for storage. The local media file may be a default factory media file, such as an instruction video included in the factory setting information of the control device. Of course, the local media file may also be obtained in other ways, and the embodiment of the present application is not limited thereto.

[0091] S104: In response to the media screen projection request, the playback device returns a parameter acquisition request to the control device.

[0092] After receiving the media screen projection request, the playback device determines that the user turns on the loading of the virtual scene on the cloud terminal device through the control device, and needs to present the stream media data of the virtual scene to the user through the playback device. At this time, in order to realize the continuous screen projection, the playback device generates a parameter acquisition request and sends the parameter acquisition request to the control device, so as to obtain specific parameters of the media file that needs to be played from the control device through the parameter acquisition request. The control device receives the parameter acquisition request that the playback device returns to the media screen projection request.

[0093] As should be explained, in the embodiment of the present application, the local media file is used to trigger the playback device to send a parameter acquisition request, that is, to trigger the screen projection receiving process flow of the playback device, but does not mean that the playback device needs to play the local media file. The media file that the control device needs to play will be transmitted to the playback device by responding to the parameter acquisition request.

[0094] S105: The control device acquires initial file parameters corresponding to the local media file in response to the parameter acquisition request returned by the playback device in response to the media screen projection request.

[0095] The control device performs parameter acquisition for the local media file in response to the parameter acquisition request. The acquired parameters are initial file parameters. As can be seen, the initial file parameters include at least a playback address corresponding to the local media file, and the playback address is generated by the control device.

[0096] In some embodiments, the initial file parameters may further include information such as a file size corresponding to the local media file, and a generation time and encoding type corresponding to the local media file, and the embodiments of the present application are not specifically limited thereto.

[0097] S106: The control device uses the stream media playback address to replace the local file address of the second protocol in the initial file parameters to obtain the target file parameters.

[0098] After obtaining the initial file parameters, the control device first locates and obtains a local file address from the initial file parameters, and then uses the stream media playback address to overwrite the local file address, thereby realizing the local file address of the second protocol being replaced with the stream media playback address of the first protocol. After completing the address replacement, the file parameters are the target file parameters that need to be sent to the playback device.

[0099] That is, in the embodiment of the present application, the control device uses the initial file parameters corresponding to the local media file as a template to generate a new target file parameter, and provides the playback address of the streaming media file that the playback device really needs to play on the screen projection in the target file parameter to the playback device, so that the target file parameter is used to provide the streaming media playback address to the playback device, and the streaming media playback address is used to provide the streaming media data of the virtual scene generated when the cloud terminal device responds to the load command.

[0100] As is necessary to explain, the local file address in the initial file parameter is used to realize the screen projection for the local media file, and the stream media playback address is used to replace the local file address. It can be understood that the stream media data corresponding to the virtual scene is presented on the playback device in the manner of video screen projection. In this way, the cloud game client of the virtual scene does not need to be involved, and the loading of the virtual scene can be realized even when the playback device does not support the cloud game client.

[0101] In some other embodiments, in addition to modifying the local file address in the initial file parameters, the control device can also modify the original size of the local media file in the initial file parameters. For example, when the size of the stream media data stored in the stream media playback address is unknown, the control device can modify the media size to infinity (the maximum value of a 64-bit number can be regarded as infinity).

[0102] As needs to be explained, the second protocol and the first protocol are different protocols. The first protocol is a protocol capable of supporting real-time transmission of stream media data, such as various stream media protocols such as RTSP, which ensures that the playback device can play the stream media data of the virtual scene in real time. The second protocol is a protocol that processes the stream media data as a file and cannot support real-time transmission, such as Hyper Text Transfer Protocol.

[0103] Illustratively, FIG. 8 is a schematic diagram of target file parameter generation provided by an embodiment of the present application. The playback device sends a parameter acquisition request 8-1 to the control device. The parameter acquisition request 8-1 includes a field 8-11 of POST / getProperty?playbackAccessLog, which indicates that the parameter acquisition request 8-1 is used to acquire specific parameters of the media file that needs to be projected on the screen, and includes a field 8-12 of Session-ID:24b3fd94, which indicates the identity of the playback device to the control device. After receiving the parameter acquisition request 8-1, the control device first acquires the initial file parameter 8-2, and replaces the content corresponding to the media size 8-21 in the initial file parameter 8-2, i.e., the content corresponding to the field duration-downloaded, from 70 to the maximum value 8-22 of the 64-bit number, and at the same time replaces the content of the link address field 8-23, i.e., the local file address, from http: / / devimages.xxx.com.xxx to the stream media playback address rtsp: / / xxxxxx 8-24. After completing all the replacement operations, the control device acquires the target file parameters.

[0104] S107: The control device returns the target file parameters to the playback device.

[0105] The playback device receives the target file parameters returned by the control device in response to the parameter get request, and analyzes and obtains a stream media playback address of the first protocol from the target file parameters returned by the control device in response to the parameter get request, thereby facilitating the playback device to subsequently pull and play the stream media data of the virtual scene from the stream media playback address.

[0106] S108: In response to the load command, the cloud terminal device loads the virtual scene, obtains stream media data corresponding to the virtual scene, and stores the stream media data at the stream media playback address.

[0107] After receiving the load command, the cloud terminal device will start to execute the virtual scene loading process, generate streaming media data when loading the virtual scene, such as video and audio when loading the virtual scene, and then directly store the obtained video and audio in the previously newly created streaming media playback address, or respectively encode and compress the audio and video to obtain a video code stream and an audio code stream, and respectively store the video code stream and the audio code stream in the streaming media playback address.

[0108] As should be explained, S108 and S104-S107, when executed, may be executed in an order other than that given in the embodiments of the present application, i.e., in some embodiments, S104-S107 may be executed before S108, or S104-S107 and S108 may be executed synchronously.

[0109] S109: The playback device pulls and obtains the stream media data of the virtual scene from the stream media playback address.

[0110] The playback device accesses the stream media playback address and pulls the stream media data stored in the stream media playback address. More specifically, the playback device first obtains the coding information of the stream media data from the stream media playback address, i.e., the coding information of the audio and video, such as the compression specification, the code rate, the video resolution, the audio sampling rate, the number of compression channels, and the number of compression bits. Then, the playback device establishes an audio channel to receive audio frames, and establishes a video channel to receive video frames, thereby receiving audio and video frame by frame and obtaining the stream media data of the virtual scene.

[0111] Illustratively, Fig. 9 is a schematic diagram of a playback device provided by an embodiment of the present application pulling stream media data. The playback device generates a data pull request 9-1 and sends the data pull request 9-1 to a data server providing stream media data. The data server first sends a request receiving prompt message to the playback device to indicate that the data pull request 9-1 has been received by the playback device, and then obtains the stream media data according to the RTSP address 9-11 (stream media playback address) in the data pull request 9-1. Fig. 10 is a schematic diagram of a request receiving prompt message provided by an embodiment of the present application, and it can be seen that in the request receiving prompt message 10-1, the field HTTP / 1.1 200 OK may be included.

[0112] S110: The playback device plays the streaming media data.

[0113] Finally, the playback device displays the video in the requested streaming media data on its own display interface, and simultaneously invokes audio output devices, such as speakers, headphones, etc., to play the audio in the streaming media data, thus completing the loading process of the virtual scene, allowing the user to interact with the virtual scene.

[0114] As can be seen, in the related art, the types of playback devices that can install the cloud gaming client are relatively few, which results in a relatively small range of devices that support the loading of the virtual scene; in the embodiment of the present application, the control device first pulls the stream media playback address from the cloud terminal device for the virtual scene, and when starting to load the virtual scene, sends a media screen projection request to the playback device for the local media file, thereby triggering the screen projection receiving processing flow of the playback device. Then, in response to the parameter acquisition request sent by the playback device, the control device modifies the local file address in the initial file parameter corresponding to the local media file to the stream media playback address of the first protocol, and returns the target file parameter to the playback device. Thus, when the playback device plays, it pulls and plays the stream media data of the virtual scene from the stream media playback address provided by the target file parameter, thereby enabling the playback device to play the stream media data of the virtual scene even when the playback device does not support the cloud gaming client, thereby realizing the loading of the virtual scene, and finally expanding the range of devices that support the loading of the virtual scene.

[0115] In some embodiments of the present application, the control device uses the stream media playback address to replace the local file address of the second protocol in the initial file parameters, and before obtaining the target file parameters, i.e., before S106, the method may further include a process in which the control device locates and obtains a link address field from the initial file parameters, and a process in which the control device locates and obtains a local file address of the second protocol from the original content corresponding to the link address field in the initial file parameters based on a protocol keyword of the second protocol.

[0116] The control device reads each field in the initial file parameters, and when the control device reads the link address field, it determines the position of the link address field in the initial file parameters, and completes the location of the link address field. Then, the control device reads the original content of the link address field, and then uses a protocol keyword of the second protocol, such as http and https, to match the original content of the link address field, thereby locating the protocol keyword in the original content. Then, the control device obtains the content after the protocol keyword, and uses the protocol keyword and the content after the protocol keyword to obtain a local file address by splicing. Here, the local file address is a playback address corresponding to a local media file, that is, if the playback device accesses the local file address, it can obtain the stream media data of the local media file.

[0117] As can be understood, the link address field may be represented using a URI or a digital identifier assigned to the link address field, and the embodiments of the present application are not specifically limited thereto.

[0118] For example, when the link address field is a uri, the control device finds the uri in the initial file parameters and retrieves the original content corresponding to the uri, for example, <string>It then locates http in the original content and takes the http and the content after http, i.e. the complete http: / / devimages.xx.com, as a local file address (i.e. the characters of the type http: / / devimages.xx.com) <string>was discarded).

[0119] The control device uses the stream media playback address to overwrite the local file address in the original content, and obtains one new file parameter after completing the overwrite operation, which is the target file parameter.

[0120] In the embodiment of the present application, the control device first locates the local file address in the initial file parameters, and then uses the stream media playback address to replace the local file address and facilitate regenerating new target file parameters, thereby realizing the target file parameters to be quickly generated by only slightly changing the initial file parameters, and thus allowing the stream media playback address to be provided to the playback device even if the playback device does not support the cloud gaming client, without the need to modify the protocol standard supported by the playback device.

[0121] In some embodiments of the present application, the control device uses the stream media playback address to replace the local file address of the second protocol in the initial file parameter. The acquisition of the target file parameter can be realized by the following process: the control device deletes the local file address from the original content corresponding to the link address field in the initial file parameter, and acquires the remaining content corresponding to the link address field; the control device inserts the stream media playback address into the remaining content based on the position information of the local file address in the original content, and acquires the latest content corresponding to the link address field; and the control device acquires the target file parameter by integrating the link address field, the latest content, other fields in the initial file parameter, and field contents corresponding to other fields.

[0122] That is, the control device deletes the local file address from the original content, and the remaining content corresponds to the link address field. Then, the control device uses the position information of the local file address in the original content, for example, the starting character of the local file address or the starting line number of the local file address, to reinsert the stream media playback address into the position corresponding to the local file address, thus obtaining the latest content corresponding to the link address field. Finally, the control device reestablishes the correspondence between the link address field and the latest content, and integrates the link address field and the latest content together with other fields other than the link address field in the initial file parameters and the field contents corresponding to the other fields to re-form the file parameters. The obtained file parameters are the target file parameters.

[0123] In an embodiment of the present application, the control device first deletes the local file address, and then inserts the stream media playback address into the original position of the local file address, thus enabling the generation of target file parameters to be realized simply and quickly, and reducing the computational resources required when generating the target file parameters.

[0124] Based on Fig. 7, refer to Fig. 11, which is a schematic diagram of another flow of a virtual scene loading method provided by an embodiment of the present application. In some embodiments of the present application, after the playback device obtains the streaming media data of the virtual scene by pulling it from the streaming media playback address, i.e., after S109, the method can further include the following steps S111 to S114:

[0125] S111: When the playback of the stream media data fails, the playback device sends a failure prompt message to the control device.

[0126] The stream media data that the playback device pulls from the stream media playback address may not be compatible with the protocol service standard of the playback device itself, and the playback device may not be able to successfully play the stream media data. In this case, the playback device will send a failure prompt message to the control device. Here, the failure prompt message is used to prompt the control device that the playback of the stream media data has failed. The control device receives the failure prompt message returned by the playback device.

[0127] In some embodiments, the playback device can autonomously trigger detection of the playback status of the stream media data to determine whether the playback of the stream media data is successful. In some other embodiments, the playback device can further trigger detection of the playback status of the stream media data when receiving a playback status callback request sent by the control device. At this time, the playback device feeds back a failure prompt message to the control device in response to the playback status callback request.

[0128] Illustratively, Figure 12 is a schematic diagram of a failure prompt message provided by an embodiment of the present application. In the failure prompt message 12-1, the data length 12-11 of the data already played by the playback device, i.e., 00000, and the playback status 12-12, i.e., paused, are included.

[0129] S112: In response to the failure prompt message, the control device requests the stream media data of the virtual scene from the stream media playback address.

[0130] After receiving the failure prompt message, the control device will determine that the playback device cannot currently play the stream media data directly, at this time, the control device will access the stream media playback address and pull and obtain the stream media data from the stream media playback address, so as to facilitate the stream media data to be transferred to the playback device after processing, and ensure that the stream media data can be normally played in the playback device.

[0131] S113: The control device converts the stream media data according to the shared service standard supported by the playback device, and obtains the converted media data.

[0132] The control device converts the requested stream media data according to the shared service standard supported by the playback device, i.e., converts the data into data that conforms to the shared service standard supported by the playback device and enables normal playback, and the media data obtained by the conversion is called converted media data.

[0133] Illustratively, when the playback device supports the DLNA standard, the control device can convert the stream media data according to the DLNA standard and obtain converted media data of the DLNA standard that can be played by the playback device.

[0134] S114: The playback device receives the converted media data that the control device replies to the failure prompt message, and plays the converted media data.

[0135] The control device transmits the converted media data to the playback device, thereby realizing the presentation of the stream media data when the virtual scene is loaded using the converted media data.

[0136] As can be seen, when the streaming media data does not conform to the protocol service standard supported by the playback device, the control device, as a "relay station" of the streaming media data, enables itself to pull the streaming media data and convert it into converted media data that conforms to the shared service standard supported by the playback device, thus contributing to improving the playback success rate of the streaming media data when loading a virtual scene, thereby further expanding the range of devices that support loading the virtual scene.

[0137] In some embodiments of the present application, after the cloud terminal device loads a virtual scene in response to the load command, obtains streaming media data corresponding to the virtual scene, and stores the streaming media data at the streaming media playback address, the method may further include: in response to a control operation on a virtual object in the virtual scene, the control device sends a control command for the virtual object to the cloud terminal device; and in response to the control command, the cloud terminal device controls the virtual object to perform a control event.

[0138] That is, the control device detects whether the user performs a control operation on the virtual object, and after detecting the control operation, generates a control command corresponding to the control operation and sends the control command to the cloud terminal device through the network. The cloud terminal device receives the control command sent by the control device to the virtual object in the virtual scene, and realizes a response to the control command by controlling the virtual object in the virtual scene. In this process, the cloud terminal device constantly generates the latest stream media data of the virtual scene and stores it in the stream media playback address, and the playback device requests the latest stream media data of the virtual scene from the stream media playback address and plays the latest stream media data.

[0139] As should be explained, the control command is used to instruct the cloud terminal device to control the virtual object to perform a control event corresponding to the control operation. The control event includes at least an event for controlling the virtual object to move in the virtual scene and an event for controlling the virtual object to interact with other objects in the virtual scene, such as controlling the virtual object to run in the virtual scene, controlling the virtual object to pick up a virtual item in the virtual scene, etc.

[0140] A control operation may refer to a user operating a physical press key or joystick on a control device, or may refer to rotating the control device beyond an angle threshold (e.g., rotating a circular haptic device beyond 90 degrees), etc., and examples of the present application are not limited thereto.

[0141] As can be understood, the control device can convert the user's control operation into a control command and then transmit it to the cloud terminal device, and the cloud terminal device can control the virtual object in the virtual scene based on the control command, thereby providing the user with the ability to interact with the virtual scene, and thereby update the stream media data of the virtual scene played by the playback device.

[0142] In some embodiments of the present application, before the cloud terminal device generates a stream media playback address of the first protocol in response to the address pull request sent by the control device, the method may further include: a process in which the control device sends an application list pull request to the cloud terminal device; a process in which the cloud terminal device responds to the application list pull request to obtain an application list corresponding to the control device and returns the application list to the control device; a process in which the control device encapsulates the application list according to the sharing service standard supported by the playback device, obtains file media, and transmits the media file to the playback device; a process in which the playback device plays the media file; and a process in which the control device screens and obtains a target application from the application list in response to a selection operation on the media file presented by the playback device, and determines an application scene corresponding to the target application as a virtual scene.

[0143] In the embodiment of the present application, different applications have different virtual scenes, and the control device needs to further present different applications to the user, so that the user can select an application and determine the virtual scene that needs to be loaded. In this situation, the control device will first send an application list pull request to the cloud terminal device, and the cloud terminal device will receive the application list pull request sent by the control device. The cloud terminal device will inquire about applications corresponding to the control device from the application library based on the device identifier of the control device contained in the application list pull request or the identity information corresponding to the user. That is, it can obtain applications that can be run on the control device, integrate these applications into an application list, and send it to the control device. The control device receives the application list returned by the cloud terminal device in response to the application list pull request. Then, the control device encapsulates the application list according to the shared service standard, obtains media files that meet the shared service standard, and sends the media files to the playback device through the network. The playback device receives the media files that meet the shared service standard sent by the control device. Finally, when the control device detects that the user has performed a selection operation on the media file, the control device will determine an application corresponding to the selected portion (e.g., an icon, a text, etc.) in the media file as a target application, and determine an application scene of the target application as a virtual scene that needs to be loaded. In this way, the control device makes it possible to determine a virtual scene that needs to be loaded.

[0144] As should be explained, the application list pull request may be triggered and generated and sent after the control device joins the network via the wireless connection module, or may be generated and sent after a user performs an operation on the control device (e.g., clicking to select a game icon) that is used to trigger the application list pull, and the embodiments of the present application are not limited thereto.

[0145] As can be understood, the application list may include cloud gaming applications, virtual reality applications (e.g., virtual dress-up applications and virtual home decoration applications) used to enhance a user's life experience running on a cloud terminal device, and 3D graphics design applications, etc., and the embodiments of the present application are not specifically limited thereto.

[0146] As can be seen, the control device packages the application list pulled from the cloud terminal device into a media file that conforms to the sharing service standard supported by the playback device, and allows the playback device to play the media file, thereby providing the application list to the user via the media file, making it easier for the user to review the application list and make application selections, while at the same time further improving the amount of information that the playback device can provide.

[0147] In some embodiments of the present application, the control device includes a display screen and a physical press key, and after the control device receives the application list returned by the cloud terminal device in response to the application list pull request, the method may further include: presenting the application list on the display screen; and screening and obtaining a target application from the application list presented by the display in response to a selection operation on the application list presented by the display, and determining an application scene corresponding to the target application as a virtual scene, where the selection operation is received by the physical press key.

[0148] 13 is a schematic diagram of the appearance of a control pad (one embodiment of a control device) provided by the present application. A display screen 13-11 and physical press keys 13-12 are provided on the control pad 13-1. After the control pad 13-1 obtains an application list, the control pad 13-1 directly presents the application list on the display screen 13-11, allowing the user to select an application. When the user performs a selection operation on the control pad 13-1 by the physical press keys 13-12, the control pad will determine a target application from among the applications provided by the application list in response to the selection operation.

[0149] As can be understood, the control device directly displays the application list on its own display screen, thereby skipping the encapsulation process for the application list, thereby saving the computational resources required to perform the encapsulation process for the application list, thereby realizing the presentation of the application list using fewer computational resources.

[0150] In some embodiments of the present application, when the control device includes a display screen and a physical press key, before the cloud terminal device generates a stream media playback address of the first protocol in response to the address pull request sent by the control device, the method can further include the following process: the control device presents on the display screen the names of multiple candidate devices in the same wireless local area network as the control device; and the control device selects a playback device from the multiple candidate devices in response to a selection operation on the multiple candidate devices by the control device.

[0151] That is, the control device obtains the names of candidate devices in the same wireless local area network and displays the obtained names on the display screen of the control device, thereby providing the user with currently available devices. Next, the control device responds to the user's device selection operation, and determines the device selected by the selection operation from among the multiple candidate devices as the playback device. In this way, it is possible to select an appropriate playback device based on the user's needs.

[0152] Of course, in some other embodiments, the control device can automatically select a playback device from among multiple candidate devices, for example, determine a device that supports the second protocol among the candidate devices as the playback device, thus realizing automatic selection of a playback device and thereby enabling a playback device to be acquired more quickly.

[0153] In some embodiments of the present application, after determining a playback device from among the multiple candidate devices in response to a selection operation for the multiple candidate devices, the method further includes a process of sending a protocol acquisition request to the playback device, a process of receiving the multiple candidate protocols replied to the protocol acquisition request by the playback device, and a process of determining a real-time stream transmission protocol from the multiple candidate protocols as a first protocol.

[0154] In the embodiment of the present application, the playback device supports at least a real-time streaming protocol and a hypertext transmission protocol, and the playback device determines all protocols it supports as a plurality of candidate protocols, whereby the plurality of candidate protocols includes at least the real-time streaming protocol and the hypertext transmission protocol. Compared with the hypertext transmission protocol, the real-time streaming protocol allows transmission in a situation where the size of the stream media data is unclear. Therefore, the control device directly determines the real-time streaming protocol as the first protocol, thereby starting the load flow of the virtual scene.

[0155] Need to explain, in some embodiments, the playback device may not support the real-time streaming transmission protocol, at this time, the control device can generate a prompt message to the playback device that "cannot load virtual scene", and display the prompt message on the display screen.

[0156] In some embodiments of the present application, the control device includes a wireless connection module, so that sending a media screen projection request corresponding to a local media file to the playback device in response to a load trigger operation on the virtual scene can be realized by the following process: sending a media screen projection request corresponding to an infinite number of local media files to the playback device according to a preset protocol standard supported by the playback device in response to a load trigger operation on the virtual scene.

[0157] Here, the preset protocol standard may be a protocol standard that is researched and developed by the brand to which the playback device belongs. As is necessary to explain, the preset protocol standard and the shared service standard are both realized through a wireless connection module. That is, the protocol stack of the preset protocol standard and the protocol stack of the control point and the media server in the shared service standard can both be realized through the wireless connection module. In this way, by modifying only the protocol supported by the control device, the streaming media data can be presented in the manner of screen projection, and the loading process for the virtual scene can be realized without modifying the protocol of the playback device.

[0158] The following describes an exemplary application of the embodiment of the present application in one practical application scenario.

[0159] The embodiment of the present application is realized in a scene where a cloud game is operated on a smart TV (playback device) that does not support a cloud game client. In the embodiment of the present application, a gamepad (control device) equipped with a WiFi module (wireless connection module) realizes the control point and media server protocol stack of the protocol service standard supported by the smart TV. An LCD screen (display screen) is added on the gamepad to display the name of the media player found in the local area network, i.e., the name of a smartphone or a smart TV, and the press key on the gamepad selects playback on the smart TV.

[0160] FIG. 14 is an operation schematic diagram of a cloud game provided by an embodiment of the present application. As shown in FIG. 14, a DLNA standard control point and a media server 14-1 are realized by a game pad 14-A, and a pad operation event 14-2 is transmitted to a server 14-B (a cloud terminal device) in an uplink channel. The server 14-B is used to operate a cloud game application 14-31 and a cloud game 14-32 (for example, game 1 to game n), generate a stream media 14-33, and process an input event 14-34. In a downlink channel, the server 14-B issues parameters 14-4 required for DLNA or a preset standard protocol to the game pad 14-A. The gamepad 14-A interacts with the smart TV 14-C based on DLNA or a preset standard protocol, and realizes a DLNA standard media player 14-5 through the smart TV 14-C to obtain and play the video and audio (stream media data) of the cloud game (virtual scene) through the real-time stream media transmission 14-6 of the RTSP protocol.

[0161] As can be seen, the gamepad can also have the functions of WiFi connection and password input. After joining the network by completing the password input, it will pull the cloud game list (application list) from the server, encapsulate the cloud game list into a media target (MediaObject, media file) in the DLNA standard according to the DLNA standard, and provide it to the media player in the smart TV for playback. In the preset protocol standard, it is necessary to encapsulate the cloud game into a media file and replace the playback address (local file address) with the stream media playback address, which provides the window screen and corresponding audio when the game scene is running.

[0162] For example, Fig. 15 is a schematic diagram of a process of encapsulating a game list provided by an embodiment of the present application into a media object in a DLNA media content service. After receiving the game list, the gamepad uses the playback address 15-11 in the detailed data 15-1 of the game object, i.e., the address of rtsp: / / xxxxx, to replace the address 15-21 in the media object 15-2 in the DLNA standard, i.e., the address of http: / / xxxxx, thereby completing the encapsulation of the game list.

[0163] 16 is a schematic diagram of replacing the playback address of the preset protocol standard provided by the embodiment of the present application with the stream media playback address provided by the cloud terminal device. The gamepad extracts the RTSP playback address 16-11 from the game object detailed data 16-1, and then uses the RTSP playback address 16-11 to replace the playback address 16-21 in the preset protocol standard 16-2 with the RTSP playback address 16-11.

[0164] When using the preset standard protocol for screen projection playback, the cloud game audio-video stream needs to be regarded as one infinite media file and played on the smart TV. The process includes the following steps 1 to 5:

[0165] Step 1: The gamepad discovers devices in the local area network according to the preset standard protocol, i.e., it notifies other devices in the local area network what services they can provide through UDP multicast packets, and at the same time discovers a smart TV that supports the preset standard protocol, and pulls the RTSP playback address (stream media playback address) from the server, which is used to transmit the screen and game audio of the cloud game running on the server.

[0166] Step 2: The game pad displays the names of devices (candidate devices) on the LCD screen, making it easy for the user to select the device to be projected onto the screen.

[0167] Step 3: The gamepad projects the infinite media file (local media file) to the smart TV screen according to the preset standard protocol. The smart TV calls the attribute acquisition interface (getProperty) in the preset standard protocol to obtain the size and playback address (local file address) of the media file. Here, the playback address will be replaced by the gamepad with the RTSP address pulled from the server in step 1.

[0168] Step 4: The gamepad calls the Play interface (load command) in the preset standard protocol to trigger the smart TV to start playing the data acquired from the RTSP address, which is a standard stream media protocol that encodes and plays audio and video to the smart TV in real time.

[0169] Step 5: If the playback is normal, the input event (control command) of the gamepad will be transmitted to the cloud terminal device, and the server will instruct it to control the character in the cloud game (perform the control event). If the playback fails, that is, as for the gamepad callback event, when the playback state is an error, switch to the DLNA standard (shared service standard) for playback.

[0170] There are several modifications to the cloud game screen played in DLNA: According to the DLNA standard, both HTTP and RTSP protocols can support media transmission, and the gamepad realizes the DLNA control point and media server. That is, the pulled game list is converted into a media file that conforms to the DLNA standard, so that it can be identified and played on a smart TV that supports the DLNA standard. The gamepad realizes the control point and media server standard in DLNA, controls the playback and operation of the cloud game as the control point, and presents the cloud game screen (converted media data) to the media player as the media server, and increases the game event control service, which can be used to adapt different pad events to different cloud games.

[0171] Illustratively, based on Fig. 1, refer to Fig. 17, which is a schematic diagram of the DLNA standard that the gamepad provided by the embodiment of the present application needs to implement. The gamepad also needs to implement a control point 1-1, a media server 1-2, and a media player 1-3, but the media server 1-2 needs to newly add and implement a game event control service 17-1.

[0172] The operation process of the control point, the media server, and the media player will be described below in conjunction with the operation process of the cloud game. Figure 18 is a schematic diagram of the operation process of the cloud game provided by an embodiment of the present application. As shown in Figure 18, the process includes the following steps S201 to S209.

[0173] S201: The control point calls a browser interface (ContentDirectory:Browser) in the content directory service of the media server to return a game list and display it on the LCD screen, and at the same time discovers and displays available playback devices (candidate devices).

[0174] As should be explained, the content directory service of the media server on the gamepad is not limited to displaying only audio-video files, but can also provide game related information, such as name, icon, introduction, etc., and can pull the information of the game list that is already installed and can be run on the server, and encapsulate it into a media object. The media object indicates to the DLNA player that it only supports RTSP protocol and does not support HTTP protocol.

[0175] S202: The control point selects the smart TV on which the playback is to be performed.

[0176] The control point calls the protocol information acquisition interface (ConnectionManager:GetProtocolInfo) in the connection management service, which returns the available device protocols and supported media formats. Typically, HTTP and RTSP protocols are returned, and the RTSP protocol is automatically selected.

[0177] S203: The control point returns a prompt that it is impossible to run the cloud game on a device that does not support the RTSP protocol.

[0178] S204: The control point calls a prepare connection interface (ConnectionManager:PrepareConnection) in the connection management service of the media server and the media player, respectively, and returns the instance ID of the smart TV.

[0179] As needs to be explained, one control point can control multiple media players, but they need to be differentiated by instance IDs. For example, a gamepad can play a game screen on multiple smart TVs.

[0180] S205: The control point calls a transport address setting interface (AVTransport:SetAVTransportURI()) in the audio video transmission service of the media server to obtain the playback address of RTSP, and calls a transport address setting interface in the audio video transmission service of the media player to set the playback address of RTSP.

[0181] S206: The control point invokes the start (AVTransport:Play()) interface in the audio video transport service to start the game.

[0182] After the cloud game is launched, the video and audio of the game screen are both compressed and made available in real time, and the audio-video stream can be stored using the rtsp: / / xxx address.

[0183] S207: The media player requests the audio-video stream of the game according to the RTSP protocol.

[0184] The media server obtains the audio / video encoding information, such as the video format, compression specification, code rate, and resolution, as well as the audio sampling rate, the number of compression channels, and the number of compression bits, based on the RTSP protocol, and then obtains the audio stream and the video stream based on the following two steps:

[0185] S2071: The media player establishes a video receiving channel based on a real-time transmission protocol (RTP) to receive video frames.

[0186] S2072: The media player establishes an audio receiving channel based on a real-time transmission protocol to receive audio frames.

[0187] S208: The control point calls the newly added game event control service in the media server to control the cloud game.

[0188] At this time, the gamepad pulls pad event configuration information from the server. For example, the directional keys can be mapped to four directions of movement or to the directional keys of a keyboard to realize control over cloud gaming.

[0189] S209: The control point calls a connection completion interface (ConnectionManager:ConnectionComplete) in the connection management service of the media server and the media player to log out from the game.

[0190] Then, if the game needs to be restarted, restart a new round of the cloud game according to the above processing steps.

[0191] As can be understood, in the embodiments of the present application, with regard to user information, such as game list, identity information, and other related data, user permission or consent must be obtained when the embodiments of the present application are implemented in a specific product or technology, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries or regions.

[0192] In the following, an exemplary structure implemented as a software module of the first device 555 for loading a virtual scene provided by the embodiment of the present application will be continuously described. In some embodiments, as shown in FIG. 4, the software module in the first device 555 for loading a virtual scene stored in the first memory 550 includes: A first receiving module 5551 configured to receive a stream media playback address of a first protocol that is returned by the cloud terminal device in response to an address pull request; A first sending module 5552 configured to send a media screen projection request corresponding to a local media file to the playback device in response to a load trigger operation on the virtual scene, and send a load command to the cloud terminal device to instruct the cloud terminal device to start loading the virtual scene; and a parameter generating module 5553 configured to obtain initial file parameters corresponding to the local media file in response to a parameter obtaining request returned by the playback device for the media screen projection request, and to use the stream media playback address to replace the local file address of the second protocol in the initial file parameters to obtain target file parameters, wherein the stream media playback address is used for providing stream media data of the virtual scene generated by the cloud terminal device when the cloud terminal device responds to the load command; The first sending module 5552 is further configured to return the target file parameters to the playback device, where the target file parameters are used for providing and playing the stream media data to the playback device.

[0193] In some embodiments of the present application, the parameter generation module 5553 is further configured to locate and obtain a link address field from the initial file parameters, and locate and obtain the local file address of the second protocol from the original content corresponding to the link address field in the initial file parameters based on a protocol keyword of the second protocol, where the local file address is a playback address corresponding to the local media file.

[0194] In some embodiments of the present application, the parameter generation module 5553 is further configured to delete the local file address from the original content corresponding to the link address field in the initial file parameters, obtain the remaining content corresponding to the link address field, insert the stream media playback address into the remaining content based on the position information of the local file address in the original content, obtain the latest content corresponding to the link address field, and integrate the link address field, the latest content, other fields in the initial file parameters, and field contents corresponding to the other fields, thereby completing the replacement of the local file address of the second protocol in the initial file parameters and obtaining the target file parameters.

[0195] In some embodiments of the present application, the first device 555 for loading the virtual scene further includes a data conversion module 5554; The first receiving module 5551 is further configured to receive a failure prompt message returned by the playback device, the failure prompt message being used to prompt that the playback of the stream media data has failed; The data conversion module 5554 is configured to request the stream media data of the virtual scene from the stream media playback address in response to the failure prompt message, convert the stream media data according to a shared service standard supported by the playback device, and obtain the converted media data; The first transmission module 5552 is further configured to transmit the converted media data to the playback device.

[0196] In some embodiments of the present application, the first transmission module 5552 is further configured to send a control command for the virtual object to the cloud terminal device in response to a control operation on the virtual object in the virtual scene, the control command is used to instruct the cloud terminal device to control the virtual object to perform a control event corresponding to the control operation, and the control event includes at least an event for controlling the virtual object to move in the virtual scene, and an event for controlling the virtual object to interact with the virtual scene.

[0197] In some embodiments of the present application, the first device 555 that loads the virtual scene further includes an information selection module 5555, and the first sending module 5552 is further configured to send an application list pull request to the cloud terminal device; The first receiving module 5551 is further configured to receive an application list returned by the cloud terminal device in response to the application list pull request; The data conversion module 5554 is further configured to encapsulate the application list, obtain media files, and transmit the media files to the playback device according to a sharing service standard supported by the playback device; The information selection module 5555 is configured to screen and acquire a target application from the application list in response to a selection operation on the media file presented by the playback device, and determine an application scene corresponding to the target application as the virtual scene.

[0198] In some embodiments of the present application, the control device includes a display screen and a physical press key, and the first device 555 that loads the virtual scene further includes a display control module 5556 configured to present the application list on the display screen; The information selection module 5555 is further configured to screen and acquire a target application from the application list presented by the display in response to a selection operation on the application list presented by the display, and determine an application scene corresponding to the target application as the virtual scene, where the selection operation is received by the physical press key.

[0199] In some embodiments of the present application, the display control module 5556 is further configured to present, on the display screen, names of a plurality of candidate devices that are in the same wireless local area network as the control device; The information selection module 5555 is further configured to determine the playback device from among the plurality of candidate devices in response to a selection operation on the plurality of candidate devices.

[0200] In some embodiments of the present application, the first sending module 5552 is further configured to send a protocol acquisition request to the playback device; The first receiving module 5551 is further configured to receive a plurality of candidate protocols returned by the playback device in response to the protocol acquisition request, the plurality of candidate protocols including at least a real-time stream transmission protocol and a hypertext transfer protocol; The information selection module 5555 is further configured to determine a real-time stream transmission protocol in the plurality of candidate protocols as the first protocol.

[0201] In some embodiments of the present application, the control device includes a wireless connection module, and the first transmission module 5552 is further configured to send the media screen projection request corresponding to an infinite number of local media files to the playback device according to a preset protocol standard supported by the playback device in response to a load trigger operation for the virtual scene, and both the preset protocol standard and the sharing service standard are realized via the wireless connection module.

[0202] The following will continue to describe an exemplary structure implemented as a software module of the second device 455 for loading a virtual scene provided by the embodiment of the present application. In some embodiments, as shown in FIG. 5, the software module in the second device 455 for loading a virtual scene stored in the second memory 450 includes: A second receiving module 4551 configured to receive a media screen projection request for a local media file sent from a control device; A second sending module 4552 configured to send a parameter acquisition request back to the control device in response to the media screen projection request; a data request module 4553 configured to: analyze and obtain a stream media playback address of a first protocol from a target file parameter returned by the control device in response to the parameter acquisition request, where the target file parameter is obtained by replacing a local file address of a second protocol in an initial file parameter with the stream media playback address, and the initial file parameter corresponds to the local media file; and pull and obtain stream media data of a virtual scene from the stream media playback address; and a data playback module 4554 configured to play the stream media data.

[0203] In some embodiments of the present application, the second sending module 4552 is further configured to send a failure prompt message to the control device when the playback of the streaming media data fails, the failure prompt message is used to prompt that the playback of the streaming media data fails; The second receiving module 4551 is further configured to receive converted media data that the control device responds to the failure prompt message; The data playback module 4554 is further configured to play the converted media data, where the converted media data is obtained by converting the stream media data requested and obtained by the control device from the stream media playback address according to a shared service standard supported by the playback device.

[0204] In some embodiments of the present application, the second receiving module 4551 is further configured to receive a media file sent by the control device, and the media file is obtained by encapsulating an application list pulled from a cloud terminal device according to a sharing service standard supported by the playback device; The data playback module 4554 is further configured to play the media file.

[0205] The following will continue to describe an exemplary structure implemented as a software module of the third device 255 for loading a virtual scene provided by the embodiment of the present application. In some embodiments, as shown in FIG. 6, the software module in the third device 255 for loading a virtual scene stored in the third memory 250 includes: An address generating module 2551 configured to generate a stream media playback address of a first protocol in response to an address pull request sent by the control device; a third sending module 2552 configured to return the stream media playback address to the control device, where the stream media playback address is used to replace a local file address of a second protocol in an initial file parameter and obtain a target file parameter, where the initial file parameter corresponds to a local media file of the control device; A third receiving module 2553 configured to receive a load command sent by the control device, the load command being used to instruct the control device to start loading the virtual scene; and a scene load module 2554 configured to, in response to the load command, load the virtual scene, obtain stream media data corresponding to the virtual scene, and store the stream media data at the stream media playback address.

[0206] In some embodiments of the present application, the third receiving module 2553 is further configured to receive a control command sent by the control device to a virtual object in the virtual scene, the control command being used to instruct a cloud terminal device to control the virtual object to perform a control event corresponding to a control operation; The scene load module 2554 is further configured to, in response to the control command, control the virtual object to perform the control events, including at least an event that controls the virtual object to move in the virtual scene and an event that controls the virtual object to interact with other objects in the virtual scene.

[0207] In some embodiments of the present application, the third receiving module 2553 is further configured to receive an application list pull request sent by the control device; The scene load module 2554 is further configured to retrieve an application list corresponding to the control device in response to the application list pull request; The third sending module 2552 is further configured to return the application list to the control device.

[0208] An embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program including computer-executable instructions, the computer-executable instructions being stored in a computer-readable storage medium, a first processor of a control device reading the computer-executable instructions from the computer-readable storage medium and executing them to realize the method for loading a virtual scene on a control device side provided by the embodiment of the present application, a second processor of a playback device reading the computer instructions from the computer-readable storage medium and executing them to realize the method for loading a virtual scene on a playback device side provided by the embodiment of the present application, and a third processor of a cloud terminal device reading the computer instructions from the computer-readable storage medium and executing them to realize the method for loading a virtual scene on a cloud terminal device side provided by the embodiment of the present application.

[0209] An embodiment of the present application provides a computer-readable storage medium on which computer-executable instructions are stored, where when the computer-executable instructions are executed by a first processor, the first processor causes the first processor to execute a virtual scene loading method on a control device side provided by an embodiment of the present application, when the computer-executable instructions are executed by a second processor, the second processor causes the second processor to execute a virtual scene loading method on a playback device side provided by an embodiment of the present application, and when executed by a third processor, the third processor causes the third processor to execute a virtual scene loading method on a cloud terminal device side provided by an embodiment of the present application.

[0210] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM, or may be any type of device including one or any combination of the above memories.

[0211] In some embodiments, the computer-executable instructions take the form of a program, software, software module, script, or code, may be written in any type of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0212] By way of example, the computer executable instructions may, but do not necessarily, correspond to a file in a file system and may be stored as part of a file that stores other programs or data, for example in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program under consideration, or in multiple shared files (e.g., files that store one or more modules, subprograms, or code portions).

[0213] As an example, the computer-executable instructions may be deployed to be executed on one computing device (a control device, a playback device, or a cloud terminal device), or on multiple computing devices located at one location, and / or on multiple computing devices distributed at multiple locations and interconnected by a communications network.

[0214] As described above, the embodiment of the present application also enables the playback of the stream media data of the virtual scene when the playback device does not support the cloud game client, realizes the loading of the virtual scene, and finally expands the range of devices that support the loading of the virtual scene. When the stream media data does not meet the protocol service standard supported by the playback device, the control device can act as a "relay station" for the stream media data, pull the stream media data by itself, and convert it into converted media data that meets the shared service standard supported by the playback device, thereby helping to improve the playback success rate of the stream media data when loading the virtual scene, thereby further improving the range of devices that support the loading of the virtual scene.

[0215] The above is merely an embodiment of the present application and is not used to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and scope of the present application are included in the scope of protection of the present application.< / string> < / string>

Claims

1. 1. A method of loading a virtual scene, executed by a control device, the method comprising: receiving a stream media playback address of a first protocol returned by the cloud terminal device in response to the address pull request; In response to a load trigger operation for the virtual scene, sending a media screen projection request corresponding to a local media file to a playback device, and sending a load command to instruct the cloud terminal device to start loading the virtual scene; obtaining initial file parameters corresponding to the local media file in response to a parameter obtainment request sent by the playback device in response to the media screen projection request; Using the stream media playback address to replace the local file address of the second protocol in the initial file parameter to obtain a target file parameter, the stream media playback address being used for providing the stream media data of the virtual scene generated when the cloud terminal device responds to the load command; returning the target file parameters to the playback device, the target file parameters being used to provide the stream media playback address to the playback device.

2. Before the step of using the stream media playback address to replace the local file address of the second protocol in the initial file parameter and obtain the target file parameter, the method further comprises: locating and obtaining a link address field from the initial file parameters; 2. The method of claim 1, further comprising: locating and obtaining the local file address of the second protocol from the original content corresponding to the link address field in the initial file parameters based on a protocol keyword of the second protocol, wherein the local file address is a playback address corresponding to the local media file.

3. Using the stream media playback address to replace the local file address of the second protocol in the initial file parameter to obtain a target file parameter, removing the local file address from the original content corresponding to the link address field in the initial file parameters, and obtaining the remaining content corresponding to the link address field; According to the location information of the local file address in the original content, insert the stream media playback address into the remaining content to obtain the latest content corresponding to the link address field; and completing replacement of the local file address of the second protocol in the initial file parameters by integrating the link address field, the latest content, other fields in the initial file parameters, and field contents corresponding to the other fields, to obtain the target file parameters.

4. After the step of returning the target file parameters to the playback device, the method further comprises: receiving a failure prompt message returned by the playback device, the failure prompt message being used to prompt that the playback of the stream media data has failed; requesting the stream media data of the virtual scene from the stream media playback address in response to the failure prompt message; converting the stream media data according to a shared service standard supported by the playback device to obtain converted media data; The method of any one of claims 1 to 3, further comprising the step of: transmitting the converted media data to the playback device.

5. After sending a load command to the cloud terminal device to start loading the virtual scene, the method further comprises: The method further includes the step of: in response to a control operation on a virtual object in the virtual scene, transmitting a control command for the virtual object to the cloud terminal device, the control command being used to instruct the cloud terminal device to control the virtual object to perform a control event corresponding to the control operation; 4. The method according to claim 1, wherein the control events include at least an event for controlling the virtual object to move in the virtual scene and an event for controlling the virtual object to interact with other objects in the virtual scene.

6. Before the step of receiving a stream media playback address of a first protocol, which is returned by the cloud terminal device in response to an address pull request, the method includes: Sending an application list pull request to the cloud terminal device; receiving an application list returned by the cloud terminal device in response to the application list pull request; encapsulating the application list, acquiring media files, and transmitting the media files to the playback device according to a shared service standard supported by the playback device; The method according to any one of claims 1 to 3, further comprising: in response to a selection operation on the media file presented by the playback device, screening and acquiring a target application from the application list, and determining an application scene corresponding to the target application as the virtual scene.

7. The control device includes a display screen and a physical press key; After receiving the application list returned by the cloud terminal device in response to the application list pull request, the method includes: presenting the list of applications on the display screen; 7. The method of claim 6, further comprising: in response to a selection operation on an application list presented by the display, screening and acquiring a target application from the application list, and determining an application scene corresponding to the target application as the virtual scene, wherein the selection operation is received by the physical press key.

8. The method comprises: presenting on the display screen names of a plurality of candidate devices that are in the same wireless local area network as the control device; The method according to claim 7 , further comprising: determining the playback device from among the plurality of candidate devices in response to a selection operation on the plurality of candidate devices.

9. After the step of determining the playback device from among the plurality of candidate devices in response to a selection operation on the plurality of candidate devices, the method further comprises: transmitting a protocol acquisition request to the playback device; receiving a plurality of candidate protocols returned by the playback device in response to the protocol acquisition request, the plurality of candidate protocols including at least a real-time stream transmission protocol and a hypertext transfer protocol; The method of claim 8 , further comprising: determining a real-time stream transmission protocol in the plurality of candidate protocols as the first protocol.

10. the control device includes a wireless connection module; sending a media screen projection request corresponding to a local media file to a playback device in response to a load trigger operation on the virtual scene; in response to a load trigger operation for the virtual scene, sending the media screen projection request corresponding to an infinite number of local media files to the playback device according to a preset protocol standard supported by the playback device; The method of claim 4 , wherein the preset protocol standard and the shared service standard are both implemented via the wireless connectivity module.

11. 1. A method for loading a virtual scene, executed by a playback device, the method comprising: receiving a media screen projection request for a local media file transmitted from a control device; returning a parameter acquisition request to the control device in response to the media screen projection request; A step of analyzing and acquiring a stream media playback address of a first protocol from the target file parameters returned by the control device in response to the parameter acquisition request, the target file parameters being acquired by replacing a local file address of a second protocol in an initial file parameter with the stream media playback address, the initial file parameter corresponding to the local media file; pulling and obtaining stream media data of the virtual scene from the stream media playback address, and playing the stream media data.

12. After pulling and obtaining the stream media data of the virtual scene from the stream media playback address, the method includes: sending a failure prompt message to the control device when the playback of the stream media data fails, the failure prompt message being used to prompt that the playback of the stream media data fails; receiving, by the control device, converted media data in response to the failure prompt message; and playing the converted media data; The method according to claim 11, wherein the converted media data is obtained by converting the stream media data requested and acquired by the control device from the stream media playback address according to a shared service standard supported by the playback device.

13. Prior to the step of receiving a media screen projection request for a local media file transmitted from a control device, the method further comprises: receiving a media file sent by the control device, the media file being obtained by encapsulating an application list pulled from a cloud terminal device according to a shared service standard supported by the playback device; The method of claim 11 , further comprising the step of: presenting the media file.

14. A method for loading a virtual scene, executed by a cloud terminal device, comprising: generating a stream media playback address of a first protocol in response to an address pull request sent by the control device; returning the stream media playback address to the control device, where the stream media playback address is used to replace a local file address of the second protocol in an initial file parameter and obtain a target file parameter, where the initial file parameter corresponds to a local media file of the control device; receiving a load command sent by the control device, the load command being used to initiate loading of the virtual scene; in response to the load command, loading the virtual scene, obtaining stream media data corresponding to the virtual scene, and storing the stream media data at the stream media playback address.

15. After the steps of loading the virtual scene in response to the load instruction, obtaining stream media data corresponding to the virtual scene, and storing the stream media data at the stream media playback address, the method further comprises: receiving a control command sent by the control device to a virtual object in the virtual scene, the control command being used to instruct a cloud terminal device to control the virtual object to perform a control event corresponding to a control operation; and controlling the virtual object to perform the control event in response to the control command. The method of claim 14 , wherein the control events include at least events that control the virtual object to move in the virtual scene and events that control the virtual object to interact with other objects in the virtual scene.

16. Before the step of generating a stream media playback address of a first protocol in response to an address pull request sent by a control device, the method further comprises: receiving an application list pull request sent by the control device; The method of claim 15 , further comprising: in response to the application list pull request, obtaining an application list corresponding to the control device, and returning the application list to the control device.

17. A first device that loads a virtual scene, the device comprising: A first receiving module is configured to receive a stream media playback address of a first protocol that is returned by a cloud terminal device in response to an address pull request; A first sending module is configured to send a media screen projection request corresponding to a local media file to a playback device in response to a load trigger operation for the virtual scene, and send a load command to the cloud terminal device to instruct the cloud terminal device to start loading the virtual scene; a parameter generation module configured to obtain initial file parameters corresponding to the local media file in response to a parameter acquisition request returned by the playback device for the media screen projection request, and to use the stream media playback address to replace the local file address of a second protocol in the initial file parameters to obtain target file parameters, wherein the stream media playback address is used for providing stream media data of the virtual scene generated by the cloud terminal device when the cloud terminal device responds to the load command; A first device for loading a virtual scene, wherein the first transmission module is further configured to return the target file parameters to the playback device, the target file parameters being used to provide the stream media playback address to the playback device.

18. a second device for loading the virtual scene, said device comprising: a second receiving module configured to receive a media screen projection request for the local media file transmitted from the control device; a second sending module configured to send a parameter acquisition request back to the control device in response to the media screen projection request; a data request module configured to: analyze and obtain a stream media playback address of a first protocol from a target file parameter returned by the control device in response to the parameter acquisition request, the target file parameter being obtained by replacing a local file address of a second protocol in an initial file parameter with the stream media playback address, the initial file parameter corresponding to the local media file; and pull and obtain stream media data of a virtual scene from the stream media playback address; a data playback module configured to play the stream media data; and a second device for loading the virtual scene.

19. a third device for loading the virtual scene, said device comprising: an address generation module configured to generate a stream media playback address of the first protocol in response to an address pull request sent by the control device; A third sending module configured to return the stream media playback address to the control device, the stream media playback address being used to replace a local file address of a second protocol in an initial file parameter and obtain a target file parameter, the initial file parameter corresponding to a local media file of the control device; a third receiving module configured to receive a load command sent by the control device, the load command being used to instruct the control device to start loading the virtual scene; and a scene load module configured to, in response to the load instruction, load the virtual scene, obtain stream media data corresponding to the virtual scene, and store the stream media data at the stream media playback address.

20. A control device, the control device comprising: a first memory adapted to store computer executable instructions; a first processor, which, when executing computer-executable instructions stored in the first memory, is used to implement the method for loading a virtual scene according to any one of claims 1 to 10.

21. A playback device, comprising: a second memory adapted to store computer executable instructions; a second processor adapted to implement the method for loading a virtual scene according to any one of claims 11 to 13 when executing computer executable instructions stored in the second memory.

22. A cloud terminal device, the cloud terminal device comprising: a third memory adapted to store computer executable instructions; and A third processor used to realize the virtual scene loading method according to any one of claims 14 to 16 when executing the computer-executable instructions stored in the third memory.

23. 15. A computer-readable storage medium having stored thereon computer-executable instructions, the computer-executable instructions being configured to, when executed by a first processor, realize the method for loading a virtual scene according to any one of claims 1 to 10, when executed by a second processor, realize the method for loading a virtual scene according to any one of claims 11 to 13, and when executed by a third processor, realize the method for loading a virtual scene according to any one of claims 14 to 16.

24. A computer program product comprising a computer program or computer executable instructions, the computer program or computer executable instructions being configured to, when executed by a first processor, implement the method for loading a virtual scene according to any one of claims 1 to 10, when executed by a second processor, implement the method for loading a virtual scene according to any one of claims 11 to 13, and when executed by a third processor, implement the method for loading a virtual scene according to any one of claims 14 to 16.

Images