Methods, devices, equipment, and computer programs for loading virtual scenes

Abstract

The present application provides a method, apparatus, device, computer-readable storage medium, and computer program product for loading a virtual scene. The embodiments of the present application can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation, and vehicle-mounted, and are related to cloud computing technology. The method includes: receiving a stream media playback address of a first protocol that a cloud terminal device responds to an address pull request; in response to a load trigger operation for the virtual scene, sending a media screen projection request corresponding to a local media file to the playback device, and sending a load command to the cloud terminal device; in response to a parameter acquisition request that the playback device responds to the media screen projection request, obtaining initial file parameters corresponding to the local media file; using the stream media playback address to replace the local file address of a second protocol in the initial file parameters to obtain target file parameters; and returning the target file parameters to the playback device.

Description

Technical Field

[0001] This application relates to cloud computing technology, and in particular to a method, apparatus, device, computer-readable storage medium, and computer program product for loading virtual scenes.

[0002] This application is proposed based on a Chinese patent application with an application number of No. 202210271200.5 and a filing date of March 18, 2022, and claims the priority of the Chinese patent application. All the contents of the Chinese patent application are incorporated herein by reference.

Background Art

[0003] Cloud game applications enable the operation of games on cloud terminal devices and the transmission of game screens and audio to playback devices via a network, thereby enabling the use of playback devices with relatively low data processing capabilities, such as smart TVs and projectors, and also enabling the loading of virtual scenes when running cloud games.

[0004] In related technologies, usually, it is necessary to first install a cloud game client on the playback device. When a player launches any cloud game, the cloud terminal device loads the virtual scene of the cloud game in the background, and the cloud game client controls the interaction between the playback device and the cloud terminal device to obtain the streaming media data of the virtual scene.

[0005] However, due to differences in data transmission protocols and hardware configurations, etc., the types of playback devices on which a cloud game client can be installed are limited, resulting in relatively few playback devices capable of loading virtual scenes, that is, the range of devices supporting the loading of virtual scenes is relatively small.

Summary of the Invention

Problems to be Solved by the Invention

[0006] Embodiments of the present application provide a method, apparatus, device, computer-readable storage medium, and computer program product for loading virtual scenes, enabling an expansion of the range of devices that support loading virtual scenes.

[0007] The technical solution of the embodiment of this application is realized as follows.

[0008] Embodiments of the present application provide a method for loading a virtual scene, which is performed by a control device, and the method is The steps include receiving the stream media playback address of the first protocol that the cloud terminal device replies to in response to an address pull request, The steps include: sending a media screen projection request corresponding to a local media file to a playback device in response to a load trigger operation for the virtual scene, and sending a load command to the cloud terminal device instructing it to start loading the virtual scene; The steps include: In response to a parameter acquisition request that the playback device sends back in response to the media screen projection request, obtaining initial file parameters corresponding to the local media file; A step of using the stream media playback address to replace the local file address of the second protocol in the initial file parameter and obtain the target file parameter, wherein the stream media playback address is used to provide the stream media data of the virtual scene generated when the cloud terminal device responds to the load command, The method includes the step of returning the target file parameters to the playback device, wherein the target file parameters are used to provide the playback device with the stream media playback address.

[0009] Embodiments of the present application provide a method for loading a virtual scene, which is performed by a playback device, and the method is The steps include receiving a media screen projection request for a local media file transmitted from a control device, The steps include: responding to the media screen projection request by sending a parameter acquisition request to the control device; A step of analyzing and obtaining the stream media playback address of the first protocol from the target file parameters returned by the control device in response to the parameter acquisition request, wherein the target file parameter is obtained by replacing the local file address of the second protocol in the initial file parameter with the stream media playback address, and the initial file parameter corresponds to the local media file. The process includes the steps of: pulling and obtaining the stream media data of a virtual scene from the stream media playback address, and playing the stream media data.

[0010] Embodiments of the present application provide a method for loading a virtual scene, which is performed by a cloud terminal device, and the method is The steps include generating a stream media playback address for the first protocol in response to an address pull request transmitted by the control device, A step of returning the stream media playback address to the control device, wherein the stream media playback address is used to replace the local file address of the second protocol in the initial file parameters and to obtain the target file parameters, the initial file parameters corresponding to the local media file of the control device, The steps include receiving a load command transmitted by the control device, which is used to instruct the device to begin loading the virtual scene, The process includes the steps of loading the virtual scene in response to the load command, acquiring stream media data corresponding to the virtual scene, and storing the stream media data at the stream media playback address.

[0011] Embodiments of the present invention provide a first device for loading a virtual scene, A first receiving module configured to receive the stream media playback address of a first protocol that a cloud terminal device replies to in response to an address pull request, A first transmission module is configured to send a media screen projection request corresponding to a local media file to a playback device and a load command to the cloud terminal device instructing it to start loading the virtual scene in response to a load trigger operation for the virtual scene. A parameter generation module configured to obtain an initial file parameter corresponding to the local media file in response to a parameter acquisition request that the playback device replies to the media screen projection request, using the stream media playback address, replacing the local file address of the second protocol in the initial file parameter, and obtaining a target file parameter, wherein the stream media playback address is used to provide the stream media data of the virtual scene generated when the cloud terminal device responds to the load command, the parameter generation module includes: The first transmission module is further configured to return the target file parameters to the playback device, and the target file parameters are used to provide the playback device with the stream media playback address.

[0012] An embodiment of the present application provides a second device for loading a virtual scene, A second receiving module configured to receive media screen projection requests for local media files transmitted from a control device, A second transmission module configured to send a parameter acquisition request back to the control device in response to the media screen projection request, A data request module is configured to: analyze and obtain the stream media playback address of the first protocol from among the target file parameters that the control device has returned in response to the parameter acquisition request, wherein the target file parameter is obtained by replacing the local file address of the second protocol in the initial file parameter with the stream media playback address, and the initial file parameter corresponds to the local media file; and pull and obtain the stream media data of the virtual scene from the stream media playback address. The system includes a data playback module configured to play the aforementioned stream media data.

[0013] Embodiments of the present application provide a third device for loading a virtual scene, An address generation module configured to generate a stream media playback address for a first protocol in response to an address pull request transmitted by a control device, A third transmission module configured to return the stream media playback address to the control device, wherein the stream media playback address is used to replace the local file address of the second protocol in the initial file parameters and to obtain the target file parameters, and the initial file parameters correspond to the local media files of the control device, A third receiving module is configured to receive a load command transmitted by the control device, which is used to instruct the device to begin loading the virtual scene. The system includes a scene load module configured to load the virtual scene in response to the load command, acquire stream media data corresponding to the virtual scene, and store the stream media data at the stream media playback address.

[0014] Embodiments of the present application provide a control device, the control device is, A first memory used to store computer-executable instructions, A first processor used to implement the method for loading a virtual scene on the control device side provided by the embodiments of the present application when executing the computer-executable instructions stored in the first memory, and includes.

[0015] The embodiments of the present application provide a playback device, and the playback device A second memory used to store computer-executable instructions, A second processor used to implement the method for loading a virtual scene on the playback device side provided by the embodiments of the present application when executing the computer-executable instructions stored in the second memory, and includes.

[0016] The embodiments of the present application provide a cloud terminal device, and the cloud terminal device A third memory used to store computer-executable instructions, A third processor used to implement the method for loading a virtual scene on the cloud terminal device side provided by the embodiments of the present application when executing the computer-executable instructions stored in the third memory, and includes.

[0017] The embodiments of the present application provide a computer-readable storage medium, on which computer-executable instructions are stored. When the computer-executable instructions are executed by the first processor, the method for loading a virtual scene on the control device side provided by the embodiments of the present application is realized. When executed by the second processor, the method for loading a virtual scene on the playback device side provided by the embodiments of the present application is realized. When executed by the third processor, the method for loading a virtual scene on the cloud terminal device side provided by the embodiments of the present application is realized.

[0018] Embodiments of the present application provide a computer program product, including a computer program or computer-executable instructions. When the computer program or computer-executable instructions are executed by a first processor, a method for loading a virtual scene on the control device provided by the embodiments of the present application is realized. When executed by a second processor, a method for loading a virtual scene on the playback device provided by the embodiments of the present application is realized. When executed by a third processor, a method for loading a virtual scene on the cloud terminal device provided by the embodiments of the present application is realized.

Advantages of the Invention

[0019] Embodiments of the present application have the following beneficial effects. For a virtual scene, the control device first pulls a streaming media playback address from a cloud terminal device, and when starting to load the virtual scene, sends a media screen projection request to the playback device for a local media file, thereby triggering the screen projection reception processing flow of the playback device. Next, in response to the parameter acquisition request sent by the playback device, the local file address in the initial file parameters corresponding to the local media file is modified to the streaming media playback address of the first protocol, and the obtained target file parameters are returned to the playback device. Thereby, when the playback device plays, it pulls and plays the streaming media data of the virtual scene from the streaming media playback address provided by the target file parameters, so that even when the playback device does not support a cloud game client, it is possible to play the streaming media data of the virtual scene, realize the loading of the virtual scene, and ultimately expand the range of devices that support the loading of the virtual scene.

Brief Description of the Drawings

[0020] [Figure 1] It is a schematic diagram of the DLNA (registered trademark) standard that the control device needs to implement. [Figure 2] It is a schematic diagram of the interface of the cloud game client. [Figure 3]This is a schematic diagram of the architecture of the virtual scene loading system provided by the embodiment of the present invention. [Figure 4] Figure 3 is a schematic diagram of the structure of the control pad provided in the embodiment of the present application. [Figure 5] Figure 3 is a schematic diagram of the structure of a smart television provided by an embodiment of the present invention. [Figure 6] Figure 3 is a schematic diagram of the server structure provided in the embodiment of the present invention. [Figure 7] This is a schematic diagram of one flow of a virtual scene loading method provided by the embodiment of the present invention. [Figure 8] This is a schematic diagram illustrating the generation of target file parameters provided by the embodiment of the present invention. [Figure 9] This is a schematic diagram illustrating how a playback device provided in the embodiment of the present invention pulls stream media data. [Figure 10] This is a schematic diagram of a request reception prompt message provided in the embodiment of the present invention. [Figure 11] This is a schematic diagram of another flow of the virtual scene loading method provided by the embodiment of the present invention. [Figure 12] This is a schematic diagram of the failure prompt message provided in the embodiment of the present invention. [Figure 13] This is a schematic diagram of the external appearance of the control pad provided in this application. [Figure 14] This is a schematic diagram illustrating the operation of a cloud game provided by an embodiment of the present invention. [Figure 15] This is a schematic diagram illustrating the process of encapsulating the game list provided by the embodiment of the present invention into a media object in a DLNA media content service. [Figure 16] This is a schematic diagram illustrating how the preset protocol standard playback address provided by the embodiment of the present invention is replaced with the stream media playback address provided by the cloud terminal device. [Figure 17] This is a schematic diagram of the DLNA standard that the gamepad provided in the embodiment of this application needs to implement. [Figure 18] This is a schematic diagram illustrating the operation process of a cloud game provided by an embodiment of the present invention. [Modes for carrying out the invention]

[0021] To further clarify the purpose, technical solution, and advantages of this application, the application is described in more detail below, along with the drawings. The embodiments described herein should not be considered limiting, and all other embodiments obtained by a person skilled in the art without creative work are all within the scope of protection of this application.

[0022] The following descriptions relate to “Several Examples,” which describe a subset of all possible examples, but “Several Examples” may be the same subset or different subsets of all possible examples, and can be combined with each other in a non-contradictory manner.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art. The terms used herein are solely for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0024] Before describing the embodiments of this application in further detail, the nouns and terms relevant to the embodiments of this application are explained below. The nouns and terms relevant to the embodiments of this application are applicable to the following interpretations.

[0025] 1) Cloud computing refers to the delivery and use mode of IT infrastructure, meaning obtaining necessary resources on demand via a network in a scalable manner. In a broader sense, cloud computing refers to the delivery and use mode of services, meaning obtaining necessary services on demand via a network in a scalable manner. Such services may be related to IT, software, and the internet, or they may be other services. Cloud computing is a product of the fusion of developments in traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0026] With the development of the internet, real-time data streams, and the diversification of connected devices, as well as the advancement of needs such as search services, social networks, mobile businesses, and open collaboration, cloud computing is developing rapidly. Unlike previous parallel distributed computing, the emergence of cloud computing is driving a revolutionary transformation of the entire internet mode and enterprise management mode from an ideological standpoint.

[0027] 2) Cloud gaming, also known as gaming on demand, is a type of online gaming technology based on cloud computing technology. Cloud gaming technology makes it possible to run high-quality games on thin client devices, which have relatively limited graphics processing and data computing capabilities. In a cloud gaming scenario, the game does not run on the player's game terminal, but on a server (for example, a cloud terminal device), and the server renders the game's virtual scene into a video and audio stream, which is then transmitted to the game terminal via the network. The game terminal does not need to have high graphics computing and data processing capabilities; it only needs to have basic stream media playback capabilities and the ability to receive player input commands and send them to the server. The playback device in the embodiment of this application can be considered a game terminal.

[0028] 3) Control devices are devices that control the startup and operation of cloud games, such as gamepads or remote controls.

[0029] 4) The Universal Plug and Play (UPnP) protocol is a type of interoperability protocol between devices that enables the provision of functions including addressing and mutual discovery of devices.

[0030] 5) The Digital Living Network Alliance (DLNA) standard is a standard established based on UPnP for devices to share content services, allowing mutual access to services provided by different devices. As needs to be explained, DLNA is a type of resolution scheme that includes multiple protocols, such as the Hypertext Transfer Protocol (HTTP) and the Hypertext Transfer Protocol over SecureSocket Layer (HTTPS).

[0031] 6) The Media Server provides media content services in accordance with the DLNA standard. The Media Server can be applied to storage devices, thereby facilitating access to picture and video content on those devices.

[0032] Media servers are used to implement standard interfaces such as Content Directory services, Connection Manager services, and AV Transport services.

[0033] Here, the content directory service is used to provide query and indexing services for media content (e.g., pictures, audio, and video), the connection management service is used to manage connections between media servers and media players, and the media transmission service standard is used to provide interfaces such as media playback addresses, playback start, and playback pause.

[0034] 7) The media player (Media Render) provides media display services in accordance with the DLNA standard. The media player can be used in playback devices such as smart TVs, projectors, and in-vehicle terminals. In other words, the playback device in the embodiment of this application supports the DLNA standard.

[0035] Media players are used to implement standard interfaces such as rendering control interfaces, connection managers, and AV transport standards.

[0036] Here, the presentation control interface is used to provide an interface for setting playback attributes (e.g., brightness, contrast, and volume), and the uses of the connection management service and media transmission service standard are as already explained in the media server section.

[0037] 8) Regarding the control point, the controller implemented in accordance with the DLNA standard is used to control the display of content by coordinating with the media server and media player. For example, a mobile terminal can implement the control point standard, thereby enabling a video app on the mobile terminal to project and play video on a playback device such as a television. The control device in the embodiment of this application can be understood as a single control point.

[0038] As needs to be explained, the control point coordinates the media server and media player by calling the standard interface implemented by the media server and the standard interface implemented by the media player.

[0039] For illustrative purposes, Figure 1 is a schematic diagram of the DLNA standard that a control device needs to implement. The control device needs to implement control point 1-1 and coordinate the operation of media server 1-2 and media player 1-3 through control point 1-1. Here, control point 1-1 includes interface application service 1-11, which is used to present useful information to the user. Media server 1-2 includes content directory service 1-21, connection management service 1-22, and media transmission service standard 1-23, media player 1-3 includes program control service 1-31, connection management service 1-32, and media transmission service standard 1-33, and control point 1-1 coordinates the operation of media server 1-2 and media player 1-3. In practice, the operation of the content directory service 1-21, connection management service 1-22, and media transmission service standard 1-23 is coordinated with the operation of the program control service 1-31, connection management service 1-32, and media transmission service standard 1-33.

[0040] 9) The Real Time Streaming Protocol (RTSP) is a protocol that encodes and transmits audio and video data in real time.

[0041] 10) "Responding to" is used to describe the conditions or states on which an operation to be performed depends. One or more operations performed when the dependent conditions or states are met may be in real time or have a set delay, and unless otherwise specified, there are no restrictions on the order in which the operations to be performed may occur.

[0042] 11) A virtual scene is a virtual scene displayed (or provided) on a display device. The virtual scene may be a simulation environment for the real world, a semi-simulated, semi-virtual virtual environment, or a purely virtual virtual environment. The virtual scene may be any one of two-dimensional, 2.5-dimensional, or three-dimensional virtual scenes. In the embodiments of this application, the virtual scene may be a virtual scene in a game application or a virtual scene in a virtual reality (VR) application.

[0043] 12) A virtual object is an image of various people and things that can be interacted with in a virtual scene, or an object that can be acted upon in a virtual scene. Such an actable object may be a virtual person, a virtual animal, or an animated person, for example, a person, an animal, a plant, a wall, or a stone that is displayed in a virtual scene. Such a virtual object may be a virtual image used to represent a single virtual user in the virtual scene. A virtual scene may contain multiple virtual objects, each of which has its own shape and volume in the virtual scene and occupies a portion of the space in the virtual scene.

[0044] Cloud gaming applications run games on servers and transmit the game screen and audio to playback devices over the network, enabling the use of playback devices with relatively low data processing capabilities, such as smart TVs, projectors, and in-car terminals, and also allowing for the loading of virtual scenes when running cloud games.

[0045] In related technologies, it is usually necessary to first install a cloud game client on the playback device. When the player (i.e., the user) launches a cloud game, the cloud game client instructs the server to load the cloud game's virtual scene in the background. The cloud game client then controls the interaction between the playback device and the server to acquire the screen and audio of the virtual scene, and controls the playback device to display the acquired screen and audio.

[0046] As needs to be explained, cloud gaming clients differ from regular game clients. While a game client runs the corresponding game locally, a cloud gaming client is equivalent to a game selection platform, allowing the user to select a different cloud game and call a server to load the selected cloud game.

[0047] For illustrative purposes, Figure 2 is a schematic diagram of the interface of a cloud game client. As shown in Figure 2, the interface 2-1 of the cloud game client includes two areas, the control options area 2-11 and the game selection area 2-12. In the control options area 2-11, basic player information 2-111 (e.g., player's nickname and rank), display control option cards 2-112, and operation option cards 2-113 (e.g., option cards for system settings and notification prompts) are presented. In the game selection area 2-12, the player selects the game they wish to launch by presenting the games 2-121 that they can choose to launch.

[0048] However, due to differences in data transmission protocols and hardware configurations, the types of playback devices on which cloud game clients can be installed are limited. For example, some brands of smart TVs cannot have cloud game clients installed. Therefore, there are relatively few playback devices that can load virtual scenes, meaning the range of devices that support loading virtual scenes is relatively small.

[0049] Embodiments of the present application provide a method, apparatus, device, computer-readable storage medium, and computer program product for loading virtual scenes, enabling an expansion of the range of devices that support loading virtual scenes. The following describes exemplary applications of the control device, playback device, and cloud terminal device provided by the embodiments of the present application. The playback device provided by the embodiments of the present application may be implemented as various types of terminals such as notebook computers, tablet computers, desktop computers, smart TVs, and in-vehicle terminals; the control device may be implemented as a control pad and haptic device; and the cloud terminal device may be implemented as a server. The following describes exemplary applications when the control device is implemented as a smart TV, the control device as a control pad, and the cloud terminal device as a server.

[0050] As shown in Figure 3, Figure 3 is a schematic diagram of the architecture of a virtual scene loading system provided by an embodiment of the present invention. In order to support a single virtual scene loading application, in the virtual scene loading system 100, a control pad 500 (control device) and a smart TV 400 (playback device) are connected to a server 200 (cloud terminal device) by a network 300, and the network 300 may be a wide area network, a local area network, or a combination of both. In the virtual scene loading system 100, a database 600 is further installed and used to provide data support to the server 200. The database 600 may be installed on the server 200 or may be independent of the server 200. Figure 3 shows a situation in which the database 600 is independent of the server 200.

[0051] The control pad 500 is used to receive the stream media playback address of the first protocol that the server 200 replies to in response to an address pull request; to send a media screen projection request to the smart TV 400 corresponding to a local media file and a load command to the server 200 instructing it to start loading the virtual scene in response to a load trigger operation for the virtual scene; to obtain initial media parameters corresponding to a local media file in response to a parameter acquisition request that the smart TV 400 replies to in response to the media screen projection request; to obtain target file parameters by replacing the local file address of the second protocol in the initial file parameters using the stream media playback address; and to return the target file parameters to the smart TV 400.

[0052] The smart TV 400 is used to receive a media screen projection request sent by the control pad 500 for a local media file, to send a parameter acquisition request back to the control pad 500 in response to the media screen projection request, to analyze and obtain the stream media playback address of the first protocol from the target file parameters returned by the control pad 500 in response to the parameter acquisition request, and to pull and obtain the stream media data of the virtual scene from the stream media playback address and to play the stream media data.

[0053] The server 200 is used to generate a stream media playback address for the first protocol in response to an address pull request sent by the control pad 500, to return the stream media playback address to the control pad 500, to receive a load command sent by the control pad 500 which is used to instruct the server to start loading a virtual scene, and in response to the load command, to load a virtual scene, to acquire stream media data corresponding to the virtual scene, and to store the stream media data at the stream media playback address.

[0054] In some embodiments, the server 200 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, secure services, CDNs, and big data and artificial intelligence platforms. The server 200, smart TV 400, and control pad 500 may be connected directly or indirectly by wired or wireless communication, and the embodiments of the present application are not limited thereto.

[0055] As shown in Figure 4, Figure 4 is a schematic diagram of the structure of a control pad (one embodiment of a control device) in Figure 3 provided by an embodiment of the present application. The control pad 500 shown in Figure 4 includes at least one first processor 510, a first memory 550, at least one first network interface 520, and a first user interface 530. Each component in the control pad 500 is coupled together by a first bus system 540. To understand this, the first bus system 540 is used to provide connection communication between these components. In addition to including a data bus, the first bus system 540 further includes a power bus, a control bus, and a status signal bus. For clarity, in Figure 4, all of the different buses are referred to as the first bus system 540.

[0056] The first processor 510 may be a type of integrated circuit chip, having signal processing capabilities, and may be, for example, a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gates, or transistor logic devices, discrete hardware components, etc. Here, the general-purpose processor may be a microprocessor or any ordinary processor, etc.

[0057] The first user interface 530 includes one or more first output devices 531 that enable the presentation of media content, one or more speakers, and / or one or more visual display screens. The first user interface 530 further includes one or more first input devices 532 that include user interface components that contribute to user input, such as a keyboard, mouse, microphone, touchscreen display screen, camera, other input buttons, and controls.

[0058] The first memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drivers, and optical disk drivers. The first memory 550 includes one or more storage devices that are selectively located away from the first processor 510 in physical location.

[0059] The first memory 550 may include volatile memory or non-volatile memory, or it may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random-access memory (RAM). The first memory 550 described in the embodiments of this application is intended to include any suitable type of memory.

[0060] In some embodiments, the first memory 550 enables various operations by storing data. Examples of this data include programs, modules, and data structures, or subsets or supersets thereof, which are described illustratively below.

[0061] The first operating system 551 includes system programs used to handle various basic system services and perform hardware-related tasks, such as a framework layer, a core library layer, and a driver layer, and is used to implement various basic operations and process hardware-based tasks. The first network communication module 552 is used to reach other computing devices via one or more (wired or wireless) first network interfaces 520, and exemplary first network interfaces 520 include Bluetooth®, Wi-Fi, and Universal Serial Bus (USB), etc. The first presentation module 553 is used to enable the presentation of information (for example, a user interface used to operate peripheral devices and to display content and information) via one or more first output devices 531 (for example, a display screen and a speaker, etc.) associated with the first user interface 530. The first input processing module 554 is used to detect one or more user inputs or interactions from one or more first input devices 532, and to translate the detected inputs or interactions.

[0062] In some embodiments, the first device for loading the virtual scene provided by the embodiments of the present invention can be implemented using a software approach. Figure 4 shows the first device 555 stored in the first memory 550. This may be software in the form of a program or plug-in, and includes software modules such as a first receiving module 5551, a first transmitting module 5552, a parameter generation module 5553, a data conversion module 5554, an information selection module 5555, and a display control module 5556. These modules are logical and can therefore be arbitrarily combined or further divided according to the implemented functions. The functions of each module are described below.

[0063] As shown in Figure 5, Figure 5 is a schematic diagram of the structure of a smart television (one embodiment of a playback device) in Figure 3 provided by an embodiment of the present application. The smart television 400 shown in Figure 5 includes at least one second processor 410, a second memory 450, at least one second network interface 420, and a second user interface 430. Each component in the smart television 400 is coupled together by a second bus system 440. To make it clear, the second bus system 440 is used to enable connection communication between these components. In addition to including a data bus, the second bus system 440 further includes a power bus, a control bus, and a status signal bus. For clarity, in Figure 5, all of the different buses are denoted as the second bus system 440.

[0064] The second processor 410 may be a type of integrated circuit chip, having signal processing capabilities, and may be, for example, a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gates, or transistor logic devices, discrete hardware components, etc. Here, the general-purpose processor may be a microprocessor or any ordinary processor, etc.

[0065] The implementation of the second user interface 430 is similar to that of the first user interface 530, and therefore will not be explained again.

[0066] In some embodiments, the second memory 450 enables various operations by storing data. Examples of this data include programs, modules, and data structures, or subsets or supersets thereof, which are described illustratively below.

[0067] The implementation of the second operating system 451, the second network communication module 452, the second presentation module 453, and the second input processing module 454 is similar to that of the first operating system 551, the first network communication module 552, the first presentation module 553, and the first input processing module 554, respectively, and will not be described again.

[0068] In some embodiments, the second device for loading the virtual scene provided by the embodiments of the present invention can be implemented using a software approach. Figure 5 shows the second device 455 stored in the second memory 450. This may be software in the form of a program or plug-in, and includes software modules: a second receiving module 4551, a second transmitting module 4552, a data request module 4553, and a data playback module 4554. These modules are logical and can therefore be arbitrarily combined or further divided according to the implemented functions. The functions of each module are described below.

[0069] As shown in Figure 6, Figure 6 is a schematic diagram of the structure of the server in Figure 3 (one embodiment of a cloud terminal device) provided by an embodiment of the present application. The server 200 shown in Figure 6 includes at least one third processor 210, a third memory 250, at least one third network interface 220, and a third user interface 230. Each component in the server 200 is coupled together by a third bus system 240. To understand this, the third bus system 240 is used to enable connection communication between these components. In addition to including a data bus, the third bus system 240 further includes a power bus, a control bus, and a status signal bus. However, for clarity, in Figure 6, all of the various buses are referred to as the third bus system 240.

[0070] The third processor 210 may be a type of integrated circuit chip, having signal processing capabilities, and may be, for example, a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gates, or transistor logic devices, discrete hardware components, etc. Here, the general-purpose processor may be a microprocessor or any ordinary processor, etc.

[0071] The implementation of the third user interface 230 is similar to that of the first user interface 530 and will not be explained again.

[0072] In some embodiments, the third memory 250 enables various operations by storing data. Examples of this data include programs, modules, and data structures, or subsets or supersets thereof, which are described illustratively below.

[0073] The implementation of the third operating system 251, the third network communication module 252, the third presentation module 253, and the third input module is similar to that of the first operating system 551, the first network communication module 552, the first presentation module 553, and the first input processing module 554, respectively, and will not be described again.

[0074] In some embodiments, the third device for loading the virtual scene provided by the embodiments of the present invention can be implemented using a software approach. Figure 6 shows the third device 255 stored in the third memory 250. This may be software in the form of a program or plug-in, and includes software modules such as an address generation module 2551, a third transmission module 2552, a third reception module 2553, and a scene loading module 2554. These modules are logical and can therefore be arbitrarily combined or further divided according to the implemented function. The function of each module will be described below.

[0075] In some embodiments, a control pad (control device), a smart TV (playback device), and a server (cloud terminal device) can implement the method of loading virtual scenes provided by embodiments of the present invention by running a computer program. For example, the computer program may be a native program or software module in an operating system, or a local application program (APP, Application), that is, a program that needs to be installed in the operating system to run, for example, a cloud game connection assistant APP, or it may be a mini-program, that is, a program that can run simply by downloading it in a browser environment, or it may be a mini-program that can be embedded in any APP. In short, the above computer program may be any form of application program, module, or plug-in.

[0076] The embodiments of this application can be applied to loading scenes of various virtual scenes, such as cloud technology, artificial intelligence, intelligent transportation, and in-vehicle applications. Below, the loading method of the virtual scene provided by the embodiments of this application will be described, along with exemplary applications and implementations of the control equipment, playback equipment, and cloud terminal equipment provided by the embodiments of this application.

[0077] As shown in Figure 7, Figure 7 is a schematic diagram of one flow of a virtual scene loading method provided by an embodiment of the present invention, and will be described in conjunction with the steps shown in Figure 7.

[0078] S101: The cloud terminal device generates a stream media playback address for the first protocol in response to an address pull request sent by the control device.

[0079] The embodiment of the present invention is implemented in a scene that loads a virtual scene. For example, by loading a virtual scene that corresponds to a cloud game, the user can have an interactive experience with the virtual scene. When loading a virtual scene is initiated, the control device first pulls the playback address of the stream media data corresponding to the virtual scene from the cloud terminal device, and then provides the playback address to the playback device, thereby facilitating the playback device to obtain and play the stream media data from the playback address. Therefore, in the embodiment of the present invention, the control device first generates an address pull request for the virtual scene and transmits the address pull request to the cloud terminal device via the network. After receiving the address pull request, the cloud terminal device generates a stream media playback address based on the first protocol, which is used to provide the stream media data generated when loading the virtual scene. In other words, in the embodiment of the present invention, the address pull request is used to pull the playback address that provides the stream media data of the virtual scene.

[0080] As needs to be explained, the first protocol may be the Real Time Streaming Protocol (RSTP) or the Real Time Messaging Protocol (RTMP). Accordingly, the format of the stream media playback address may be rtsp: / / xxxxxx or rtmp: / / xxxxx.

[0081] To make it understandable, the stream media data may include video footage of the virtual scene, for example, changes in the appearance of the surrounding scene as a virtual object walks, and may further include audio for the virtual scene, for example, sound effects of flowing water in the virtual scene.

[0082] To further clarify, the virtual scene may be a virtual scene in a cloud gaming application or a virtual reality application, and the embodiments of this application are not limited thereto. The address pull request includes an identifier for the application to which the virtual scene belongs, and the cloud terminal device parses and obtains this identifier from the address pull request and determines the virtual scene for which it is necessary to generate the corresponding stream media playback address based on the identifier.

[0083] In the embodiment of the present invention, a wireless connection module, such as a Wi-Fi module, can be installed in the control device, thereby enabling the control device to communicate with cloud terminal devices by joining a network via the wireless connection module.

[0084] S102: The cloud terminal device sends the stream media playback address back to the control device.

[0085] The cloud terminal device completes its response to the address pull request by generating a stream media playback address and then transmitting it to the control device via the network. The control device receives the stream media playback address of the first protocol that the cloud terminal device replies to the address pull request. In other words, the stream media playback address is the one that the cloud terminal device replies to the address pull request it sent.

[0086] As needs to be explained, S101 and S102 may be executed and completed before receiving the virtual scene load trigger operation. That is, the control device first obtains the stream media playback address from the cloud terminal device and then uses it directly when it receives the virtual scene load trigger operation. In this case, the address pull request in S101 may be generated and sent by the control device in response to an additional operation by the user, where an additional operation refers to an operation to add an application corresponding to the virtual scene to the application library corresponding to the user. For example, when a user adds a game to "My Games", an address pull request is generated and sent to the cloud terminal device. S101 and S102 may be executed in response to the virtual scene load trigger operation. That is, when the control device receives the user's load trigger operation for the virtual scene, it may synchronously pull the stream media playback address from the cloud terminal device and send a load command, but this is not specifically limited here. In this embodiment of the present invention, the virtual scene loading method of this embodiment will be explained using the example that the process of S101-S102 is executed first, and then the response to the load trigger operation for the virtual scene is started.

[0087] S103: In response to a load trigger operation for the virtual scene, the control device sends a media screen projection request corresponding to the local media file to the playback device, and also sends a load command to the cloud terminal device instructing it to start loading the virtual scene.

[0088] The control device detects in real time whether the user has performed a load trigger operation on the virtual scene. When a load trigger operation is detected, it acquires local media files by performing a local file scan. It also generates a media screen projection request for the local media files and sends it to the playback device, thereby calling the playback device to perform screen projection. The playback device receives the media screen projection request for the local media files sent from the control device. Simultaneously, the control device can also generate a load command used to instruct the cloud terminal device to start loading the virtual scene and send the load command to the cloud terminal device over the network. The cloud terminal device receives the load command sent by the control device, which is used to instruct the cloud terminal device to start loading the virtual scene.

[0089] To make it clear, detection of a load trigger operation can be completed by whether or not a physical press key on the control device is operated by the user. For example, when the user presses the start key on the control pad (one implementation of the control device), the control pad determines that a load trigger operation has been detected. Detection of a load trigger operation can also be completed by whether or not the control device is moved. For example, when the user shakes a tactile device (another embodiment of the control device) and reaches an angle threshold, the tactile device determines that a load trigger operation has been detected. Embodiments of the present application are not specifically limited herein.

[0090] The local media file may be a history media file stored in the control device. For example, when a cloud terminal device receives a stop command for a virtual scene on the control device within a historical time (used to instruct the cloud terminal device to stop loading to the virtual scene), the stream media data of the virtual scene from 1 second prior to receiving the stop command is saved, a history media file is obtained, and it is sent to the control device for storage. The local media file may be a default factory default media file, such as an instructional video included in the factory default settings information of the control device. Of course, the local media file may be obtained by other means, and the embodiments of this application are not limited thereto.

[0091] S104: In response to the media screen projection request, the playback device sends a parameter acquisition request back to the control device.

[0092] After receiving a media screen projection request, the playback device must determine that the user has enabled the loading of the virtual scene on the cloud terminal device via the control device, and then present the virtual scene's stream media data to the user. At this time, in order to continuously enable screen projection, the playback device generates a parameter acquisition request and sends it to the control device, thereby obtaining the specific parameters of the media files that need to be played back from the control device. The control device receives the parameter acquisition request that the playback device has sent back in response to the media screen projection request.

[0093] As needs to be explained, in the embodiments of this application, the local media file is used to trigger the playback device to send a parameter acquisition request. That is, it is used to trigger the screen projection reception processing flow of the playback device, and does not mean that the playback device needs to play the local media file. The media file that the control device needs to play will be transmitted to the playback device in response to the parameter acquisition request.

[0094] S105: The control device responds to the parameter acquisition request that the playback device replies to the media screen projection request by acquiring the initial file parameters corresponding to the local media file.

[0095] The control device retrieves parameters for the local media file in response to a parameter retrieval request. The retrieved parameters are the initial file parameters. To make it clear, the initial file parameters include at least a playback address corresponding to the local media file, which is generated by the control device.

[0096] In some embodiments, the initial file parameters may further include information such as the file size corresponding to the local media file, and the generation time and encoding type corresponding to the local media file, and the embodiments of the present invention are not specifically limited herein.

[0097] S106: The control device uses the stream media playback address to replace the local file address of the second protocol in the initial file parameters and obtains the target file parameters.

[0098] The control device, after obtaining the initial file parameters, first locates and obtains the local file address from the initial file parameters, then uses the stream media playback address to overwrite the local file address, thereby replacing the local file address of the second protocol with the stream media playback address of the first protocol. The file parameters after address replacement are the target file parameters that need to be sent to the playback device.

[0099] In other words, in the embodiment of the present invention, the control device uses the initial file parameters corresponding to the local media file as a template to generate a new target file parameter, and in the target file parameter, provides the playback device with the playback address of the stream media file that the playback device truly needs to perform screen projection playback of. Therefore, the target file parameter is used to provide the playback device with the stream media playback address, and the stream media playback address is used to provide the stream media data of the virtual scene generated when the cloud terminal device responds to the load command.

[0100] As needs to be explained, the local file address in the initial file parameters is used to enable screen projection of the local media file, thereby replacing the local file address with the stream media playback address. In practice, this means that the stream media data corresponding to the virtual scene is presented on the playback device using a video screen projection method. In this way, the cloud game client for the virtual scene does not need to be involved, and the virtual scene can be loaded even when the playback device does not support the cloud game client.

[0101] In some other embodiments, the control device can also modify the original size of the local media file in the initial file parameters, in addition to modifying the local file address in the initial file parameters. For example, when the size of the stream media data stored in the stream media playback address is unknown, the media size can be modified to infinity (the maximum value of a 64-bit number can be considered infinity).

[0102] As needs to be explained, the second protocol and the first protocol are different protocols. The first protocol is a protocol that enables support for real-time transmission of stream media data. For example, various stream media protocols such as RTSP ensure that playback devices can play back stream media data of virtual scenes in real time. The second protocol is a protocol that processes stream media data as a file and cannot support real-time transmission, such as the Hyper Text Transfer Protocol.

[0103] Illustratively, Figure 8 is a schematic diagram illustrating the generation of target file parameters provided by an embodiment of the present invention. The playback device sends a parameter acquisition request 8-1 to the control device. The parameter acquisition request 8-1 includes the field 8-11 POST / getProperty?playbackAccessLog, indicating that the parameter acquisition request 8-1 is used to acquire specific parameters of the media file that needs to be screen projected, and the field 8-12 Session-ID:24b3fd94, indicating the identity of the playback device to the control device. After receiving the parameter acquisition request 8-1, the control device first acquires the initial file parameters 8-2, replaces the content corresponding to the media size 8-21 in the initial file parameters 8-2, i.e., the content corresponding to the field duration-downloaded, from 70 to the maximum value of 64 bits 8-22, and simultaneously replaces the content of the link address field 8-23, i.e., the local file address, from http: / / devimages.xxx.com.xxx to the stream media playback address rtsp: / / xxxxxx8-24. After all replacement operations are complete, the control device obtains the target file parameters.

[0104] S107: The control device returns the target file parameters to the playback device.

[0105] The playback device receives the target file parameters that the control device has returned in response to the parameter acquisition request, and parses and obtains the stream media playback address of the first protocol from the target file parameters that the control device has returned in response to the parameter acquisition request, thereby facilitating the subsequent pulling of the virtual scene's stream media data from the stream media playback address and playback.

[0106] S108: In response to a load command, the cloud terminal device loads a virtual scene, acquires the stream media data corresponding to the virtual scene, and stores the stream media data at the stream media playback address.

[0107] After receiving a load command, the cloud terminal device begins the virtual scene loading process and generates stream media data for loading the virtual scene, such as video and audio. The acquired video and audio are then either directly stored in the previously created stream media playback address, or the audio and video are encoded and compressed to obtain a video code stream and an audio code stream, which are then stored in the stream media playback address.

[0108] As needs to be explained, S108 and S104-S107 may be executed in an order other than the order given in the embodiments of the present application when executed. That is, in some embodiments, S104-S107 may be executed before S108, or S104-S107 and S108 may be executed synchronously.

[0109] S109: The playback device retrieves the stream media data of the virtual scene from the stream media playback address.

[0110] The playback device accesses the stream media playback address and pulls the stream media data stored at that address. More specifically, the playback device first obtains the encoding information of the stream media data from the stream media playback address, i.e., the encoding information of audio and video, which includes information such as compression specifications, code rate, video resolution, audio sampling rate, number of compressed channels, and number of compressed bits. Next, it establishes an audio channel to receive audio frames and a video channel to receive video frames, thereby receiving audio and video frame by frame and obtaining the stream media data for the virtual scene.

[0111] Illustratively, Figure 9 is a schematic diagram of a playback device provided in an embodiment of the present invention pulling stream media data. The playback device generates a data pull request 9-1 and sends the data pull request 9-1 to a data server that provides stream media data. The data server first sends a request reception prompt message to the playback device to indicate that it has received the data pull request 9-1, and then retrieves the stream media data according to the RTSP address 9-11 (stream media playback address) in the data pull request 9-1. Figure 10 is a schematic diagram of a request reception prompt message provided in an embodiment of the present invention, and it can be seen that the field HTTP / 1.1 200 OK may be used in the request reception prompt message 10-1.

[0112] S110: The playback device plays back stream media data.

[0113] Finally, the playback device displays the video from the requested stream media data on its own display interface and simultaneously invokes audio output devices, such as speakers and headphones, to play the audio from the stream media data. In this way, the user interacts with the virtual scene by completing the virtual scene loading process.

[0114] To make it easier to understand, in related technologies, the types of playback devices that can install cloud game clients are relatively few, which limits the range of devices that support loading virtual scenes. In contrast, in the embodiment of the present invention, the control device first pulls a stream media playback address from the cloud terminal device for the virtual scene, and when it starts loading the virtual scene, it sends a media screen projection request to the playback device for the local media file, thereby triggering the screen projection reception processing flow of the playback device. Next, in response to the parameter acquisition request sent by the playback device, it modifies the local file address in the initial file parameter corresponding to the local media file to the stream media playback address of the first protocol, and returns the target file parameter to the playback device. This allows the playback device to pull and play the stream media data of the virtual scene from the stream media playback address provided by the target file parameter when it plays, thereby enabling playback of the virtual scene's stream media data even when the playback device does not support cloud game clients, thus realizing the loading of virtual scenes and ultimately expanding the range of devices that support loading virtual scenes.

[0115] In some embodiments of the present invention, the control device may use the stream media playback address to replace the local file address of the second protocol in the initial file parameters, and before obtaining the target file parameters, i.e., before S106, the method may further include the process by which the control device locates and obtains the link address field from the initial file parameters, and the process by which the control device locates and obtains the local file address of the second protocol from the original content corresponding to the link address field in the initial file parameters, based on the protocol keyword of the second protocol.

[0116] The control device reads each field in the initial file parameters, and when it reads the link address field, it determines the position of the link address field in the initial file parameters, thus completing the positioning of the link address field. Next, the control device reads the original content of the link address field and then uses the protocol keywords of the second protocol, such as http and https, to match the original content of the link address field and locate the protocol keywords within the original content. Next, it obtains the content following the protocol keywords and uses the protocol keywords and the content following them to splice and obtain the local file address. Here, the local file address is the playback address corresponding to the local media file; that is, if the playback device accesses the local file address, it can obtain the stream media data of the local media file.

[0117] To ensure clarity, the link address field may be represented using a URI, or it may be represented using a digital identifier assigned to the link address field, and the embodiments of this application are not specifically limited herein.

[0118] For example, when the link address field is URI, the control device finds the URI in the initial file parameters and retrieves the original content corresponding to the URI, for example, <string>This will result in obtaining http: / / devimages.xx.com. Next, locate http in the original content and take the content after http, i.e., the complete http: / / devimages.xx.com, as the local file address (i.e., the type of characters http: / / devimages.xx.com). <string>(It was discarded.)

[0119] The control device utilizes the stream media playback address to overwrite the local file address in the original content, and after completing the overwrite operation, it is able to obtain a new file parameter, which is the target file parameter.

[0120] In the embodiment of the present invention, the control device first locates the local file address from the initial file parameters, thereby facilitating the replacement of the local file address using the stream media playback address and regenerating the new target file parameters. This enables the rapid generation of target file parameters by only slightly changing the initial file parameters, and thus allows the stream media playback address to be provided to the playback device without the need to modify the protocol standards supported by the playback device, even if the playback device does not support cloud game clients.

[0121] In some embodiments of the present invention, the control device utilizes the stream media playback address to replace the local file address of the second protocol in the initial file parameter. Obtaining the target file parameter can be achieved by the control device removing the local file address from the original content corresponding to the link address field in the initial file parameter and obtaining the remaining content corresponding to the link address field; the control device inserting the stream media playback address into the remaining content based on the positional information of the local file address in the original content and obtaining the latest content corresponding to the link address field; and the control device obtaining the target file parameter by integrating the link address field, the latest content, other fields in the initial file parameter, and the field content corresponding to the other fields.

[0122] In other words, the control device takes the original content from which the local file address has been removed and makes it the remaining content corresponding to the link address field. Next, the control device uses the positional information of the local file address in the original content, such as the starting character of the local file address or the starting line number of the local file address, to reinsert the stream media playback address at the position corresponding to the local file address, and thus obtains the latest content corresponding to the link address field. Finally, the control device re-establishes the correspondence between the link address field and the latest content, and integrates the link address field and the latest content together with the other fields in the initial file parameters other than the link address field, and the field content corresponding to those other fields, to reshape the file parameters. The obtained file parameters are the target file parameters.

[0123] In the embodiment of the present invention, the control device first deletes the local file address and then inserts the stream media playback address into the original position of the local file address, thereby enabling the generation of target file parameters to be simple and rapid, and reducing the computational resources required when generating target file parameters.

[0124] Based on Figure 7, and as referenced in Figure 11, Figure 11 is a schematic diagram of another flow of a method for loading a virtual scene provided by an embodiment of the present application. In some embodiments of the present application, after the playback device obtains the stream media data of the virtual scene from the stream media playback address, i.e., after S109, the method may further include the following steps S111 to S114.

[0125] S111: When playback of stream media data fails, the playback device sends a failure prompt message to the control device.

[0126] The stream media data pulled by the playback device from the stream media playback address may not be playable by the playback device due to reasons such as not conforming to the playback device's own protocol service standards. In this case, the playback device will send a failure prompt message to the control device. Here, the failure prompt message is used to prompt the control device that playback of the stream media data has failed. The control device receives the failure prompt message sent back by the playback device.

[0127] In some embodiments, the playback device can determine whether or not playback of the stream media data is successful by autonomously triggering detection of the playback status of the stream media data. In some other embodiments, the playback device can further trigger detection of the playback status of the stream media data when it receives a playback status callback request sent by the control device. In this case, the playback device feeds back a failure prompt message to the control device in response to the playback status callback request.

[0128] Illustratively, Figure 12 is a schematic diagram of a failure prompt message provided by an embodiment of the present invention. Failure prompt message 12-1 includes the data length 12-11 that the playback device has already played back, i.e., 00000, and the playback status 12-12, i.e., paused.

[0129] S112: In response to the failure prompt message, the control device requests the stream media data for the virtual scene from the stream media playback address.

[0130] After receiving a failure prompt message, the control device determines that the playback device is currently unable to directly play the stream media data. At this point, the control device accesses the stream media playback address and retrieves the stream media data from it. This facilitates the processing of the stream media data before transferring it to the playback device, ensuring that the stream media data can be successfully played back on the playback device.

[0131] S113: The control device converts the stream media data according to the shared service standard supported by the playback device and obtains the converted media data.

[0132] The control device converts the requested stream media data according to the shared service standard supported by the playback device. That is, it converts the data into data that conforms to the shared service standard supported by the playback device and enables normal playback, and the converted media data is designated as the converted media data.

[0133] For example, when a playback device supports the DLNA standard, a control device can convert the stream media data according to the DLNA standard and obtain DLNA-standard converted media data that the playback device can play.

[0134] S114: The playback device receives the converted media data that the control device sends back in response to the failure prompt message, and plays the converted media data.

[0135] The control device transmits the converted media data to the playback device, thereby enabling the presentation of the stream media data when loading a virtual scene using the converted media data.

[0136] To make it easier to understand, when stream media data does not conform to the protocol service standards supported by the playback device, the control device acts as a "relay station" for the stream media data, pulling the stream media data itself and converting it into converted media data that conforms to the shared service standards supported by the playback device. In this way, it contributes to improving the success rate of streaming media data playback when loading virtual scenes, thereby further expanding the range of devices that support loading virtual scenes.

[0137] In some embodiments of the present invention, the cloud terminal device loads a virtual scene in response to a load command, acquires stream media data corresponding to the virtual scene, and stores the stream media data at a stream media playback address. The method may further include the process by which the control device sends a control command to the cloud terminal device for a virtual object in response to a control operation on the virtual object in the virtual scene, and the process by which the cloud terminal device controls the virtual object and performs a control event in response to the control command.

[0138] In other words, the control device detects whether the user has performed a control operation on a virtual object, and after detecting the operation, generates a control command corresponding to the operation and transmits the control command to the cloud terminal device via the network. The cloud terminal device receives the control command that the control device transmits to the virtual object in the virtual scene and responds to the control command by controlling the virtual object in the virtual scene. In this process, the cloud terminal device constantly generates the latest stream media data for the virtual scene and stores it in the stream media playback address, and the playback device requests the latest stream media data for the virtual scene from the stream media playback address and plays the latest stream media data.

[0139] As needs to be explained, control commands are used to instruct cloud terminal devices to control virtual objects and perform control events corresponding to control operations. Control events include, at a minimum, events that control virtual objects to move in a virtual scene, and events that control virtual objects to interact with other objects in the virtual scene, such as controlling a virtual object to run in a virtual scene, or controlling a virtual object to pick up a virtual item in a virtual scene.

[0140] Control operation may refer to the user operating physical press keys and joysticks on the control device, or it may refer to rotating the control device beyond an angular threshold (for example, rotating a ring-shaped tactile device beyond 90 degrees), and the embodiments of this application are not limited thereto.

[0141] To understand this, the control device can transmit the user's control operations to the cloud terminal device by converting them into control commands. The cloud terminal device controls virtual objects in the virtual scene based on the control commands, thereby providing the user with the ability to interact with the virtual scene. This then updates the stream media data of the virtual scene played back by the playback device.

[0142] In some embodiments of the present invention, the cloud terminal device, in response to an address pull request sent by the control device, before generating a stream media playback address for the first protocol, may further include: the process of the control device sending an application list pull request to the cloud terminal device; the process of the cloud terminal device responding to the application list pull request by obtaining an application list corresponding to the control device and returning the application list to the control device; the process of the control device encapsulating the application list, acquiring file media, and transmitting the media file to the playback device according to the shared service standard supported by the playback device; the process of the playback device playing the media file; and the process of the control device responding to a selection operation for a media file presented by the playback device by screening and acquiring a target application from the application list and determining an application scene corresponding to the target application as a virtual scene.

[0143] In the embodiment of the present invention, different applications have different virtual scenes, and the control device further needs to present different applications to the user, so the user makes an application selection and thereby determines the virtual scene that needs to be loaded. In this situation, the control device first sends an application list pull request to the cloud terminal device, and the cloud terminal device receives the application list pull request sent by the control device. Based on the device identifier of the control device or the identity information corresponding to the user included in the application list pull request, the cloud terminal device queries the application library for applications that correspond to the control device. That is, it can obtain applications that can run on the control device, integrate these applications into an application list, and send it to the control device. The control device receives the application list that the cloud terminal device has returned in response to the application list pull request. Subsequently, the control device encapsulates the application list according to the shared service standard, obtains a media file that conforms to the shared service standard, and sends the media file to the playback device over the network. The playback device receives the media file that conforms to the shared service standard sent by the control device. Finally, when the control device detects that the user has made a selection operation on a media file, it determines the application corresponding to the selected portion of the media file (e.g., icons and text) as the target application, and determines the application scene of the target application as the virtual scene that needs to be loaded. In this way, the control device is able to determine the virtual scene that needs to be loaded.

[0144] As needs to be explained, an application list pull request may be triggered, generated, and transmitted after the control device joins the network via a wireless connection module, or it may be generated and transmitted after the user performs an operation used to trigger an application list pull on the control device (for example, clicking and selecting a game icon), and the embodiments of the present invention are not limited thereto.

[0145] To ensure clarity, the application list may include cloud gaming applications, virtual reality applications that run on cloud terminal devices and are used to enhance the user's life experience (e.g., virtual dress-up applications and virtual home decor applications), and 3D graphics design applications, and the embodiments of this application are not specifically limited herein.

[0146] To make it easier to understand, the control device packages the application list pulled from the cloud terminal device into a media file that conforms to the shared service standards supported by the playback device, and the playback device can play the media file. By providing the application list to the user via the media file, it makes it easier for the user to review the application list and select an application, while simultaneously increasing the amount of information that the playback device can provide.

[0147] In some embodiments of the present invention, the control device includes a display screen and a physical press key. In this case, after the control device receives an application list returned by a cloud terminal device in response to an application list pull request, the method may further include the process of presenting the application list on the display screen, and the process of screening and acquiring a target application from the application list in response to a selection operation on the application list presented by the display, and determining an application scene corresponding to the target application as a virtual scene, wherein the selection operation is received by a physical press key.

[0148] Illustratively, Figure 13 is a schematic diagram of the external appearance of a control pad (one embodiment of a control device) provided by the present invention. A display screen 13-1 and physical press keys 13-12 are installed on the control pad 13-1. After acquiring an application list, the control pad 13-1 can allow the user to select an application by directly presenting the application list on the display screen 13-11. When the user performs a selection operation on the control pad 13-1 using the physical press keys 13-12, the control pad will respond to the selection operation by determining a target application from the applications provided in the application list.

[0149] To make it easier to understand, the control device can skip the encapsulation process for the application list by displaying it directly on its own display screen, thereby saving the computational resources that would normally be required for encapsulation. This allows for the presentation of the application list using fewer computational resources.

[0150] In some embodiments of the present invention, when the control device includes a display screen and a physical press key, the cloud terminal device may, in response to an address pull request transmitted by the control device, before generating a stream media playback address for the first protocol, further include the process of the control device presenting the names of a plurality of candidate devices on the display screen in the same wireless local area network as the control device, and the process of the control device selecting a playback device from the plurality of candidate devices in response to a selection operation for the plurality of candidate devices.

[0151] In other words, the control device obtains the names of candidate devices on the same wireless local area network and displays the obtained names on the control device's display screen, thereby providing the user with currently available devices. Next, the control device responds to the user's device selection operation and determines the device selected by the user from among several candidate devices as the playback device. In this way, it is possible to select an appropriate playback device based on the user's needs.

[0152] Of course, in some other embodiments, the control device can automatically select a playback device from among multiple candidate devices, for example, by determining that a device supporting the second protocol is the playback device. In this way, automated selection of playback devices is achieved, thereby enabling faster acquisition of playback devices.

[0153] In some embodiments of the present invention, in response to a selection operation for a plurality of candidate devices, after determining a playback device from among the plurality of candidate devices, the method further includes the process of sending a protocol acquisition request to the playback device, the process of receiving a plurality of candidate protocols that the playback device replies to the protocol acquisition request, and the process of determining a real-time stream transmission protocol among the plurality of candidate protocols as the first protocol.

[0154] In the embodiments of the present invention, the playback device supports at least a real-time stream transmission protocol and a hypertext transmission protocol, and the playback device determines all the protocols it supports as multiple candidate protocols. Thus, the multiple candidate protocols include at least a real-time stream transmission protocol and a hypertext transmission protocol. The real-time stream transmission protocol enables transmission in situations where the size of the stream media data is unclear, compared to the hypertext transmission protocol. Therefore, the control device initiates the virtual scene load flow by directly determining the real-time stream transmission protocol as the first protocol.

[0155] As needs to be explained, in some embodiments, the playback device may not support the real-time stream transmission protocol. In this case, the control device can generate a prompt message to the playback device stating "Unable to load virtual scene" and display this prompt message on the display screen.

[0156] In some embodiments of the present invention, the control device includes a wireless connection module. This makes it possible to send a media screen projection request corresponding to a local media file to a playback device in response to a load trigger operation on a virtual scene by the following process: In response to a load trigger operation on a virtual scene, a media screen projection request corresponding to an infinite number of local media files is sent to the playback device in accordance with a preset protocol standard supported by the playback device.

[0157] Here, the preset protocol standard may be a protocol standard developed and researched by the brand to which the playback device belongs. As needs to be explained, both the preset protocol standard and the shared service standard are implemented via a wireless connection module. That is, both the protocol stack of the preset protocol standard and the protocol stacks of the control point and media server in the shared service standard can be implemented by a wireless connection module. In this way, by modifying only the protocols supported by the control device, stream media data can be presented in a screen projection manner, eliminating the need to modify the playback device's protocol and enabling loading processing for virtual scenes.

[0158] The following describes an exemplary application of one embodiment of the present invention in a real-world application scenario.

[0159] The embodiment of this invention is implemented in a scenario where cloud gaming is enabled on a smart TV (playback device) that does not support cloud game clients. In the embodiment of this invention, a gamepad (control device) equipped with a WiFi module (wireless connection module) enables the smart TV to use a standard protocol service control point and media server protocol stack. An LCD screen (display screen) is added to the gamepad to display the name of a media player discovered on the local area network, i.e., the name of a smartphone or smart TV, and the user selects to play the content on the smart TV by pressing a key on the gamepad.

[0160] Figure 14 is a schematic diagram of the operation of a cloud game provided by an embodiment of the present invention. As shown in Figure 14, the gamepad 14-A implements a DLNA standard control point and media server 14-1, and transmits gamepad operation events 14-2 to server 14-B (cloud terminal device) on the uplink channel. Server 14-B is used to operate the cloud game application 14-31 and cloud games 14-32 (for example, games 1 to n), generate stream media 14-33, and process input events 14-34. On the downlink channel, server 14-B issues parameters 14-4 required for DLNA or a preset standard protocol to the gamepad 14-A. The gamepad 14-A interacts with the smart TV 14-C based on DLNA or a preset standard protocol, and the smart TV 14-C enables a DLNA standard media player 14-5, which acquires and plays video and audio (stream media data) of cloud games (virtual scenes) via real-time stream media transmission 14-6 of the RTSP protocol.

[0161] To make it easier to understand, the gamepad can also have WiFi connectivity and password entry capabilities. After joining the network by completing password entry, it pulls the cloud game list (application list) from the server, encapsulates the cloud game list in a media target (MediaObject, media file) within the DLNA standard, and provides it to the media player on the smart TV for playback. In the preset protocol standard, the cloud game needs to be encapsulated in a single media file, and the playback address (local file address) needs to be replaced with a stream media playback address, which provides the window screen when the game scene is running, and the corresponding audio.

[0162] Illustratively, Figure 15 is a schematic diagram illustrating the process of encapsulating a game list provided by an embodiment of the present invention into a media object in a DLNA media content service. After receiving the game list, the gamepad uses the playback address 15-11 in the game object's detailed data 15-1, i.e., the address rtsp: / / xxxxx, to replace the address 15-21 in the media object 15-2 within the DLNA standard, i.e., the address http: / / xxxxx, thereby completing the encapsulation of the game list.

[0163] Figure 16 is a schematic diagram illustrating how the preset protocol standard playback address provided by the embodiment of the present invention is replaced with the stream media playback address provided by the cloud terminal device. The gamepad extracts the RTSP playback address 16-11 from the detailed data 16-1 of the game object, and then uses the RTSP playback address 16-11 to replace the playback address 16-21 in the preset protocol standard 16-2 with the RTSP playback address 16-11.

[0164] When performing screen projection playback using a preset standard protocol, the audio and video stream of a cloud game needs to be treated as a single, infinite media file for playback on the smart TV. This process includes steps 1 through 5 below.

[0165] Step 1: The gamepad discovers the device on the local area network according to the preset standard protocol. That is, it notifies other devices on the local area network via UDP multicast packets what services they can provide, and at the same time discovers smart TVs that support the preset standard protocol, pulls the RTSP playback address (stream media playback address) from the server, which is used to transmit the screen and game audio of the cloud game running on the server.

[0166] Step 2: The gamepad displays the name of the device (candidate device) on the LCD screen, making it easier for the user to select the device they want to project onto the screen.

[0167] Step 3: The gamepad projects an infinite amount of media files (local media files) onto the smart TV screen according to the preset standard protocol. The smart TV calls the getProperty interface in the preset standard protocol to obtain the size and playback address (local file address) of the media files. Here, the playback address will be replaced by the RTSP address pulled from the server in Step 1 by the gamepad.

[0168] Step 4: The gamepad invokes the Play interface (load command) in a preset standard protocol, triggering the smart TV to begin playback of data obtained from the RTSP address. This address is a standard stream media protocol that encodes and plays audio and video to the smart TV in real time.

[0169] Step 5: If playback is successful, the gamepad input events (control commands) are transmitted to the cloud terminal device, instructing the server to control the character in the cloud game (perform control events). If playback fails, i.e., if the gamepad callback event indicates an error in the playback state, the system switches to the DLNA standard (shared service standard) for playback.

[0170] The following are some points regarding the modification of cloud game screens played via DLNA. According to the DLNA standard, both the HTTP and RTSP protocols can support media transmission, and the gamepad can act as both a DLNA control point and a media server. That is, by converting the pulled game list into media files that conform to the DLNA standard, they can be identified and played on smart TVs that support the DLNA standard. The gamepad acts as both a control point and a media server in DLNA, controlling the playback and operation of cloud games as a control point, and presenting the cloud game screen (converted media data) to the media player as a media server, and enhancing the Game Event Control service, which is used to adapt different gamepad events for different cloud games.

[0171] For example, as shown in Figure 17, based on Figure 1, Figure 17 is a schematic diagram of the DLNA standard that the gamepad provided by the embodiment of the present invention needs to implement. The gamepad also needs to implement a control point 1-1, a media server 1-2, and a media player 1-3, but the media server 1-2 needs to implement a newly added game event control service 17-1.

[0172] The following describes the operation process of the control point, media server, and media player in conjunction with the operation process of the cloud game. Figure 18 is a schematic diagram of the operation process of the cloud game provided by the embodiment of the present application. As shown in Figure 18, the process includes the following steps S201 to S209.

[0173] S201: The control point calls the browser interface (ContentDirectory:Browser) in the media server's content directory service to return a game list and display it on the LCD screen, while simultaneously discovering and displaying available playback devices (candidate devices).

[0174] As needs to be explained, the media server's content directory service on the gamepad is not limited to displaying only audio and video files, but can also provide game-related information, such as the name, icon, and description of games already installed on the server and playable, and encapsulate this information within a Media Object. The Media Object indicates that it supports only the RTSP protocol to the DLNA player and does not support the HTTP protocol.

[0175] S202: The control point selects the smart TV you want to play from.

[0176] The control point calls the ConnectionManager:GetProtocolInfo interface in the connection management service, which returns the protocols and supported media formats of available devices. It typically returns HTTP and RTSP protocols, and automatically selects the RTSP protocol.

[0177] S203: The control point returns a prompt indicating that it is impossible to run the cloud game on a device that does not support the RTSP protocol.

[0178] S204: The control point calls the ConnectionManager:PrepareConnection interface in the media server and media player connection management service, respectively, and returns the smart TV instance ID.

[0179] As needs to be explained, one control point can control multiple media players, but they need to be distinguished by their instance ID. That is, a gamepad can play game footage on multiple smart TVs.

[0180] S205: The control point obtains the RTSP playback address by calling the transmission address setting interface (AVTransport:SetAVTransportURI()) in the media server's audio-video transmission service, and sets the RTSP playback address by calling the transmission address setting interface in the media player's audio-video transmission service.

[0181] S206: The control point starts the game by calling the start (AVTransport:Play()) interface in the audio-video transmission service.

[0182] After a cloud game is launched, the video and audio from the game screen are both compressed and made available in real time, and the audio-video stream can be stored using the rtsp: / / xxx address.

[0183] S207: The media player requests the game's audio and video streams according to the RTSP protocol.

[0184] The media server obtains audio and video encoding information based on the RTSP protocol, such as video format, compression specifications, code rate, and resolution, as well as audio sampling rate, number of compressed channels, and number of compressed bits. Subsequently, it obtains audio and video streams based on the following two steps.

[0185] S2071: The media player establishes a video receiving channel based on the Real-Time Transmission Protocol (RTP) and receives video frames.

[0186] S2072: The media player establishes an audio receiving channel based on a real-time transmission protocol and receives audio frames.

[0187] S208: The control point calls a newly added game event control service on the media server to control the cloud game.

[0188] At this time, the gamepad pulls the server's pad event configuration information. For example, by mapping the directional keys to movement in four directions, or by mapping them to the keyboard's directional keys, control over the cloud game can be achieved.

[0189] S209: The control point calls the Connection Manager: ConnectionComplete interface in the media server and media player connection management service to log out of the game.

[0190] Afterward, if it is necessary to restart the game, restart the cloud game for a new round according to the processing steps described above.

[0191] To ensure understanding, in the embodiments of this application, with regard to user information, such as game lists and personal information, and other related data, user permission or consent must be obtained when the embodiments of this application are implemented in a specific product or technology. Furthermore, the collection, use, and processing of such related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0192] Hereinafter, exemplary structures implemented as software modules of the first device 555 that load virtual scenes provided by the embodiments of the present application will be described. In some embodiments, as shown in Figure 4, the software module in the first device 555 that loads virtual scenes stored in the first memory 550 is: A first receiving module 5551 is configured to receive the stream media playback address of the first protocol that the cloud terminal device replies to in response to an address pull request, A first transmission module 5552 is configured to send a media screen projection request corresponding to a local media file to a playback device in response to a load trigger operation for the virtual scene, and to send a load command to the cloud terminal device instructing it to start loading the virtual scene. A parameter generation module 5553 is configured to obtain initial file parameters corresponding to the local media file in response to a parameter acquisition request that the playback device replies to the media screen projection request, to use the stream media playback address, replace the local file address of the second protocol in the initial file parameters, and obtain target file parameters, wherein the stream media playback address is used to provide the stream media data of the virtual scene generated when the cloud terminal device responds to the load command, and the parameter generation module 5553 may include such a module, The first transmission module 5552 is further configured to return the target file parameters to the playback device, and the target file parameters are used to provide and play the stream media data to the playback device.

[0193] In some embodiments of the present invention, the parameter generation module 5553 is further configured to locate and obtain a link address field from the initial file parameters, and to locate and obtain the local file address of the second protocol from the original content corresponding to the link address field in the initial file parameters, based on the protocol keyword of the second protocol, wherein the local file address is the playback address corresponding to the local media file.

[0194] In some embodiments of the present invention, the parameter generation module 5553 is configured to further remove the local file address from the original content corresponding to the link address field in the initial file parameter, obtain the remaining content corresponding to the link address field, insert the stream media playback address into the remaining content based on the location information of the local file address in the original content, obtain the latest content corresponding to the link address field, and complete the replacement of the local file address for the second protocol in the initial file parameter by integrating the link address field, the latest content, other fields in the initial file parameter, and the field content corresponding to the other fields, thereby obtaining the target file parameter.

[0195] In some embodiments of the present application, the first device 555 for loading the virtual scene further includes a data conversion module 5554. The first receiving module 5551 described above is further configured to receive a failure prompt message returned by the playback device, and the failure prompt message is used to prompt that playback of the stream media data has failed. The data conversion module 5554 is configured to respond to the failure prompt message by requesting the stream media data of the virtual scene from the stream media playback address, converting the stream media data according to the shared service standard supported by the playback device, and obtaining the converted media data. The first transmission module 5552 described above is further configured to transmit the converted media data to the playback device.

[0196] In some embodiments of the present invention, the first transmission module 5552 is further configured to send control commands to the cloud terminal device for the virtual object in response to a control operation on the virtual object in the virtual scene, the control commands being used to instruct the cloud terminal device to control the virtual object and perform control events corresponding to the control operation, the control events including at least an event that controls the virtual object to move in the virtual scene, and an event that controls the virtual object to interact with the virtual scene.

[0197] In some embodiments of the present invention, the first device 555 for loading the virtual scene further includes an information selection module 5555, and the first transmission module 5552 is further configured to send an application list pull request to the cloud terminal device. The first receiving module 5551 described above is further configured to receive the application list that the cloud terminal device has sent back in response to the application list pull request. The above data conversion module 5554 is further configured to encapsulate the above application list, acquire media files, and transmit the above media files to the above playback device, in accordance with the shared service standard supported by the above playback device. The information selection module 5555 is configured to respond to a selection operation for the media file presented by the playback device by screening and acquiring a target application from the application list, and to determine the application scene corresponding to the target application as the virtual scene.

[0198] In some embodiments of the present application, the control device includes a display screen and physical press keys, and the first device 555 for loading the virtual scene further includes a display control module 5556 configured to display the application list on the display screen. The information selection module 5555 is further configured to screen and acquire a target application from the application list presented by the display in response to a selection operation, and to determine the application scene corresponding to the target application as the virtual scene, and the selection operation is received by the physical press key.

[0199] In some embodiments of the present invention, the display control module 5556 is further configured to display on the display screen the names of a plurality of candidate devices located on the same wireless local area network as the control device. The information selection module 5555 described above is further configured to determine the playback device from among the multiple candidate devices in response to a selection operation for the multiple candidate devices.

[0200] In some embodiments of the present invention, the first transmission module 5552 is further configured to transmit a protocol acquisition request to the playback device. The first receiving module 5551 described above is further configured to receive a plurality of candidate protocols that the playback device replies to the protocol acquisition request, wherein the plurality of candidate protocols include at least a real-time stream transmission protocol and a hypertext transmission protocol. The information selection module 5555 described above is further configured to determine the real-time stream transmission protocol among the multiple candidate protocols described above as the first protocol.

[0201] In some embodiments of the present invention, the control device includes a wireless connection module, the first transmitting module 5552 is further configured to transmit to the playback device a media screen projection request corresponding to an infinite number of local media files in response to a load trigger operation for the virtual scene, in accordance with a preset protocol standard supported by the playback device, both of which are implemented via the wireless connection module.

[0202] Hereinafter, exemplary structures implemented as software modules for the second device 455 that load virtual scenes provided by the embodiments of the present application will be described. In some embodiments, as shown in Figure 5, the software module in the second device 455 that loads virtual scenes stored in the second memory 450 is: A second receiving module 4551 is configured to receive media screen projection requests for local media files transmitted from a control device, A second transmission module 4552 is configured to respond to the above media screen projection request by sending a parameter acquisition request to the above control device, The data request module 4553 is configured to analyze and obtain the stream media playback address of the first protocol from the target file parameters that the control device has returned in response to the parameter acquisition request, wherein the target file parameters are obtained by replacing the local file address of the second protocol in the initial file parameters with the stream media playback address, and the initial file parameters correspond to the local media file, and to pull and obtain the stream media data of the virtual scene from the stream media playback address. It may include a data playback module 4554 configured to play the above-mentioned stream media data.

[0203] In some embodiments of the present application, the second transmission module 4552 is further configured to send a failure prompt message to the control device when playback of the stream media data fails, and the failure prompt message is used to prompt that playback of the stream media data has failed. The second receiving module 4551 described above is further configured to receive the conversion media data that the control device sends back in response to the failure prompt message. The data playback module 4554 described above is further configured to play the converted media data, which is obtained by converting the stream media data requested and obtained by the control device from the stream media playback address, in accordance with the shared service standard supported by the playback device.

[0204] In some embodiments of the present invention, the second receiving module 4551 is further configured to receive media files transmitted by the control device, the media files being obtained by encapsulating an application list pulled from a cloud terminal device, in accordance with the shared service standard supported by the playback device. The data playback module 4554 described above is further configured to play the media files described above.

[0205] Hereinafter, exemplary structures implemented as software modules of the third device 255 that load virtual scenes provided by the embodiments of the present application will be described. In some embodiments, as shown in Figure 6, the software module in the third device 255 that loads virtual scenes stored in the third memory 250 is: An address generation module 2551 is configured to generate a stream media playback address for the first protocol in response to an address pull request transmitted by a control device, A third transmission module 2552 configured to return the above-mentioned stream media playback address to the above-mentioned control device, wherein the above-mentioned stream media playback address is used to replace the local file address of the second protocol in the initial file parameter and to obtain the target file parameter, and the above-mentioned initial file parameter corresponds to the local media file of the above-mentioned control device, A third receiving module 2553 is configured to receive a load command transmitted by the control device described above, which is used to instruct the device to begin loading the virtual scene described above. The system may include a scene load module 2554 configured to load the virtual scene in response to the load command, acquire stream media data corresponding to the virtual scene, and store the stream media data at the stream media playback address.

[0206] In some embodiments of the present application, the third receiving module 2553 is further configured to receive control commands transmitted by the control device to the virtual object in the virtual scene, and the control commands are used to instruct the cloud terminal device to control the virtual object and perform control events corresponding to control operations. The scene load module 2554 is further configured to control the virtual object and perform the control events in response to the control commands, the control events including at least an event that controls the virtual object to move in the virtual scene, and an event that controls the virtual object to interact with other objects in the virtual scene.

[0207] In some embodiments of the present application, the third receiving module 2553 is further configured to receive application list pull requests transmitted by the control device. The scene load module 2554 is further configured to retrieve an application list corresponding to the control device in response to the application list pull request. The third transmission module 2552 described above is further configured to return the application list to the control device.

[0208] The embodiment of the present application provides a computer program product or computer program, which includes computer executable instructions, and which are stored in a computer-readable storage medium. The first processor of the control device reads and executes the computer executable instructions from the computer-readable storage medium to realize the method of loading a virtual scene on the control device side provided by the embodiment of the present application, the second processor of the playback device reads and executes the computer instructions from the computer-readable storage medium to realize the method of loading a virtual scene on the playback device side provided by the embodiment of the present application, and the third processor of the cloud terminal device reads and executes the computer instructions from the computer-readable storage medium to realize the method of loading a virtual scene on the cloud terminal device side provided by the embodiment of the present application.

[0209] Embodiments of the present application provide a computer-readable storage medium in which computer-executable instructions are stored. When a computer-executable instruction is executed by a first processor, the first processor causes the control device side virtual scene loading method provided by embodiments of the present application to be executed. When the instruction is executed by a second processor, the second processor causes the playback device side virtual scene loading method provided by embodiments of the present application to be executed. When the instruction is executed by a third processor, the third processor causes the cloud terminal device side virtual scene loading method provided by embodiments of the present application to be executed.

[0210] In some embodiments, the computer-readable storage medium may be memory such as FRAM®, ROM, PROM, EPROM, EEPROM, flash memory, surface-mount memory, optical disk, or CD-ROM, or it may be various devices containing one or any combination of the above-mentioned memories.

[0211] In some embodiments, computer executable instructions may take the form of a program, software, software module, script, or code, and may be written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as a standalone program, or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0212] For example, computer executable instructions may, but do not necessarily, correspond to files in a file system; they may be stored as part of other programs or files that store data. For instance, they may be stored in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program under consideration, or in multiple collaborative files (for example, files that store one or more modules, subprograms, or code sections).

[0213] For example, computer executable instructions may be executed on one computing device (a control device, a playback device, or a cloud terminal device), or on multiple computing devices located in one place, or on multiple computing devices distributed in multiple locations and interconnected by a communication network.

[0214] As described above, the embodiment of the present invention makes it possible to play stream media data of a virtual scene even when the playback device does not support the cloud game client, thereby enabling the loading of the virtual scene and ultimately expanding the range of devices that support the loading of the virtual scene. When the stream media data does not conform to the protocol service standards supported by the playback device, the control device acts as a "relay station" for the stream media data, pulling the stream media data itself and converting it into converted media data that conforms to the shared service standards supported by the playback device, thereby helping to improve the success rate of playing the stream media data when loading the virtual scene, and further expanding the range of devices that support the loading of the virtual scene.

[0215] The foregoing are merely examples of the present invention and are not intended to limit the scope of protection. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of the present invention are all included within the scope of protection.< / string> < / string>

Claims

1. A method for loading a virtual scene, which is performed by a control device, wherein the method is: The steps include receiving the stream media playback address of the first protocol that the cloud terminal device replies to in response to an address pull request, The steps include: sending a media screen projection request corresponding to a local media file to a playback device in response to a load trigger operation for the virtual scene, and sending a load command to the cloud terminal device instructing it to start loading the virtual scene; The steps include: In response to a parameter acquisition request that the playback device sends back in response to the media screen projection request, obtaining initial file parameters corresponding to the local media file; A step of replacing the local file address of the second protocol in the initial file parameters with the stream media playback address and obtaining the target file parameters, wherein the stream media playback address is used to provide the stream media data of the virtual scene generated when the cloud terminal device responds to the load command, A method for loading a virtual scene, comprising the steps of: returning the target file parameters to the playback device, wherein the target file parameters are used to provide the playback device with the stream media playback address.

2. Before the step of replacing the local file address of the second protocol in the initial file parameters with the stream media playback address and obtaining the target file parameters, the method: The steps include: locating and obtaining the link address field from the aforementioned initial file parameters; The method according to claim 1, further comprising the step of locating and obtaining the local file address of the second protocol from the original content corresponding to the link address field in the initial file parameters, based on the protocol keyword of the second protocol, wherein the local file address is a playback address corresponding to the local media file.

3. The step of replacing the local file address of the second protocol in the initial file parameters with the stream media playback address and obtaining the target file parameters is as follows: The steps include: removing the local file address from the original content corresponding to the link address field in the initial file parameters, and obtaining the remaining content corresponding to the link address field; The steps include: inserting the stream media playback address into the remaining content based on the location information of the local file address in the original content, and obtaining the latest content corresponding to the link address field; The method of claim 2, comprising the step of integrating the link address field, the latest content, other fields in the initial file parameters, and field content corresponding to the other fields to complete replacing the local file address of the second protocol in the initial file parameters with the stream media playback address, and obtaining the target file parameters.

4. After the step of returning the target file parameters to the playback device, the method The steps include receiving a failure prompt message returned by the playback device, wherein the failure prompt message is used to prompt that playback of the stream media data has failed, In response to the failure prompt message, the steps include requesting the stream media data of the virtual scene from the stream media playback address, The steps include: converting the stream media data according to the shared service standard supported by the playback device and obtaining the converted media data; The method according to claim 1, further comprising the step of transmitting the converted media data to the playback device.

5. After sending a load command to the cloud terminal device instructing it to start loading the virtual scene, the method: A step of sending a control command to a cloud terminal device for a virtual object in the virtual scene in response to a control operation on the virtual object, the control command being used to instruct the cloud terminal device to control the virtual object and perform a control event corresponding to the control operation, further comprising the step of The method according to claim 1, wherein the control events include at least an event that controls the virtual object to move in the virtual scene, and an event that controls the virtual object to interact with other objects in the virtual scene.

6. Prior to the step in which the cloud terminal device receives the stream media playback address of the first protocol that it replies to in response to the address pull request, the method The steps include sending an application list pull request to the aforementioned cloud terminal device, The steps include receiving the application list that the cloud terminal device has returned in response to the application list pull request, The steps include encapsulating the application list, acquiring media files, and transmitting the media files to the playback device, in accordance with the shared service standard supported by the playback device. The method according to claim 1, further comprising the steps of: in response to a selection operation for the media file presented by the playback device, screening and acquiring a target application from the application list, and determining an application scene corresponding to the target application as the virtual scene.

7. The control device includes a display screen and physical press keys. After the cloud terminal device receives the application list that it has returned in response to the application list pull request, the method: The steps include presenting the application list on the display screen, The method according to claim 6, further comprising the steps of: in response to a selection operation for an application list presented on the display screen, screening and acquiring a target application from the application list, and determining an application scene corresponding to the target application as the virtual scene, wherein the selection operation is received by pressing a physical key.

8. The aforementioned method, The steps include displaying the names of multiple candidate devices located on the same wireless local area network as the control device on the display screen, The method according to claim 7, further comprising the step of determining the playback device from among the plurality of candidate devices in response to a selection operation for a plurality of candidate devices.

9. After the step of determining the playback device from among the multiple candidate devices in response to a selection operation for the multiple candidate devices, the method: The steps include sending a protocol acquisition request to the playback device, A step of receiving a plurality of candidate protocols that the playback device replies to the protocol acquisition request, wherein the plurality of candidate protocols include at least a real-time stream transmission protocol and a hypertext transmission protocol. The method according to claim 8, further comprising the step of determining a real-time stream transmission protocol among a plurality of candidate protocols as the first protocol.

10. The control device includes a wireless connection module, Sending a media screen projection request corresponding to a local media file to the playback device in response to a load trigger operation for the virtual scene is: The step of sending a media screen projection request to the playback device, corresponding to an infinite number of local media files, in response to a load trigger operation for the virtual scene, according to a preset protocol standard supported by the playback device, The method according to claim 4, wherein both the preset protocol standard and the shared service standard are implemented via the wireless connection module.

11. A method for loading a virtual scene, which is performed by a playback device, wherein the method is: The steps include receiving a media screen projection request for a local media file transmitted from a control device, The steps include: responding to the media screen projection request by sending a parameter acquisition request to the control device; A step of analyzing and obtaining the stream media playback address of the first protocol from the target file parameters returned by the control device in response to the parameter acquisition request, wherein the target file parameter is obtained by replacing the local file address of the second protocol in the initial file parameter with the stream media playback address, and the initial file parameter corresponds to the local media file. A method for loading a virtual scene, comprising the steps of: pulling and obtaining stream media data of a virtual scene from the stream media playback address, and playing the stream media data.

12. After pulling and obtaining the stream media data of the virtual scene from the aforementioned stream media playback address, the method then: A step of sending a failure prompt message to the control device when playback of the stream media data fails, wherein the failure prompt message is used to prompt that playback of the stream media data has failed. The method further includes the step of receiving the converted media data that the control device replies to the failure prompt message, and playing back the converted media data, The method according to claim 11, wherein the converted media data is obtained by converting the stream media data requested and obtained by the control device from the stream media playback address in accordance with the shared service standard supported by the playback device.

13. Prior to the step of receiving a media screen projection request for a local media file transmitted from a control device, the method: The step of receiving a media file transmitted by the control device, wherein the media file is obtained by encapsulating an application list pulled from a cloud terminal device in accordance with the shared service standard supported by the playback device. The method according to claim 11, further comprising the step of presenting the media file.

14. A method for loading a virtual scene, which is executed by a cloud terminal device, wherein the method is: The steps include generating a stream media playback address for the first protocol in response to an address pull request transmitted by the control device, A step of returning the stream media playback address to the control device, wherein the target file parameter is obtained by replacing the local file address of the second protocol in the initial file parameter with the stream media playback address, and the initial file parameter corresponds to the local media file of the control device. The steps include receiving a load command transmitted by the control device, which is used to instruct the device to begin loading the virtual scene, A method for loading a virtual scene, comprising the steps of loading the virtual scene in response to the load command, acquiring stream media data corresponding to the virtual scene, and storing the stream media data at the stream media playback address.

15. After the steps of loading the virtual scene in response to the load command, acquiring stream media data corresponding to the virtual scene, and storing the stream media data at the stream media playback address, the method: The steps include: receiving a control command transmitted by the control device to a virtual object in the virtual scene, wherein the control command is used to instruct the cloud terminal device to control the virtual object and perform a control event corresponding to the control operation; The process further includes the step of controlling the virtual object to perform the control event in response to the control command, The method according to claim 14, wherein the control events include at least an event that controls the virtual object to move in the virtual scene, and an event that controls the virtual object to interact with other objects in the virtual scene.

16. Prior to the step of generating a stream media playback address for the first protocol in response to an address pull request transmitted by the control device, the method: The steps include receiving an application list pull request transmitted by the control device, The method according to claim 15, further comprising the steps of obtaining an application list corresponding to the control device in response to the application list pull request and returning the application list to the control device.

17. A first device for loading a virtual scene, wherein the device is A first receiving module configured to receive the stream media playback address of a first protocol that a cloud terminal device replies to in response to an address pull request, A first transmission module is configured to send a media screen projection request corresponding to a local media file to a playback device and a load command to the cloud terminal device instructing it to start loading the virtual scene in response to a load trigger operation for the virtual scene, A parameter generation module configured to obtain initial file parameters corresponding to the local media file in response to a parameter acquisition request that the playback device replies to the media screen projection request, replace the local file address of the second protocol in the initial file parameters with the stream media playback address, and obtain target file parameters, wherein the stream media playback address is used to provide the stream media data of the virtual scene generated when the cloud terminal device responds to the load command, the parameter generation module includes: The first transmission module is further configured to return the target file parameters to the playback device, the target file parameters being used to provide the playback device with the stream media playback address, the first device for loading a virtual scene.

18. A second device for loading a virtual scene, wherein the device is A second receiving module configured to receive media screen projection requests for local media files transmitted from a control device, A second transmission module configured to send a parameter acquisition request back to the control device in response to the media screen projection request, A data request module is configured to: analyze and obtain the stream media playback address of the first protocol from among the target file parameters that the control device has returned in response to the parameter acquisition request, wherein the target file parameter is obtained by replacing the local file address of the second protocol in the initial file parameter with the stream media playback address, and the initial file parameter corresponds to the local media file; and pull and obtain the stream media data of the virtual scene from the stream media playback address. A second device for loading a virtual scene, comprising a data playback module configured to play the aforementioned stream media data.

19. A third device for loading a virtual scene, wherein the device is An address generation module configured to generate a stream media playback address for a first protocol in response to an address pull request transmitted by a control device, A third transmission module configured to return the stream media playback address to the control device, which is used to replace the local file address of the second protocol in the initial file parameters with the stream media playback address and to obtain target file parameters, wherein the initial file parameters correspond to the local media files of the control device, A third receiving module is configured to receive a load command transmitted by the control device, which is used to instruct the device to begin loading the virtual scene. A third device for loading a virtual scene, comprising: a scene load module configured to load the virtual scene in response to the load command, acquire stream media data corresponding to the virtual scene, and store the stream media data at the stream media playback address.

20. A control device, wherein the control device is A first memory used to store computer executable instructions, A control device comprising: a first processor used to implement a virtual scene loading method according to any one of claims 1 to 10 when executing a computer executable instruction stored in the first memory.

21. A playback device, wherein the playback device is A second memory used to store computer executable instructions, A playback device comprising: a second processor used to implement the method for loading a virtual scene according to any one of claims 11 to 13 when executing a computer executable instruction stored in the second memory.

22. A cloud terminal device, wherein the cloud terminal device is A third memory used to store computer executable instructions, A cloud terminal device comprising: a third processor used to implement the virtual scene loading method described in any one of claims 14 to 16 when executing computer executable instructions stored in the third memory.

23. A computer program configured to cause a computer to perform the method for loading a virtual scene described in any one of claims 1 to 10.

24. A computer program configured to cause a computer to perform the method for loading a virtual scene described in any one of claims 11 to 13.

25. A computer program configured to cause a computer to perform the method for loading a virtual scene described in any one of claims 14 to 16.

Images