Interaction method, system, device and equipment based on cloud game, medium and product
By using control devices and cloud servers to collect physical environment data in cloud games and simulate player behavior, the problem of poor interactivity in cloud games is solved, and the game experience and player retention rate are improved.
Patent Information
- Application Number
- CN202310589304.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2024-11-26
AI Technical Summary
In the existing cloud gaming technology, the interaction between players and cloud gaming is poor, with one-way feedback and a lack of effective two-way interactive experience.
By introducing control devices and cloud servers in cloud games, the interactive behavior of virtual objects and physical environment data collection are realized, the behavior of players in the physical environment is simulated, and the interaction between players and cloud game characters is enhanced.
It improves the interaction between players and cloud games, improves game experience and player retention, simulates player behavior in cloud games through virtual objects, and enhances the authenticity and interactivity of game feedback.
Smart Images

Figure CN119015709A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of cloud gaming, and in particular to an interactive method, system, device, equipment, medium and product based on cloud gaming. Background Art
[0002] Cloud gaming is a gaming method based on cloud computing. In the cloud gaming mode, all games are run on the server side, and the server side will transmit the rendered game screen to the display device through the network for display.
[0003] In the related art, cloud games display the game screen through a display device, and control the game through a control device (such as a mobile phone, a game controller, etc.). When playing a game, if a player triggers a highlight moment, a special effect animation is usually displayed on the display device or a vibration control device is used to prompt the player.
[0004] However, in the related art, when the player triggers a highlight moment, the feedback given by the cloud game to the player is one-way, and the interactivity between the player and the cloud game is poor. Summary of the invention
[0005] The embodiments of the present application provide a cloud game-based interactive method, system, device, equipment, medium, and product, which can enhance the interactivity between players and cloud games. The technical solution is as follows:
[0006] In one aspect, a cloud gaming-based interactive method is provided, which is performed by a display device, and the method includes:
[0007] Displaying a first virtual object controlled by a control device in the cloud game, wherein the first virtual object performs a first virtual behavior in the cloud game, and the first virtual behavior is a behavior performed based on a control control in the control device;
[0008] When the first virtual behavior meets the interaction requirement, displaying the first virtual object to perform a second virtual behavior;
[0009] The second virtual behavior is a behavior generated based on the activity data collected by the control device after collecting activity data of the entity object in the physical environment, and the second virtual behavior is used to simulate the behavior of the entity object in the physical environment.
[0010] On the other hand, a cloud game-based interactive method is provided, which is performed by a control device, and the method includes:
[0011] Receiving a first behavior control operation on the control device, and sending a first behavior control instruction to the cloud server, where the first behavior control instruction is used to control a first virtual object in the cloud game to perform a first virtual behavior;
[0012] When the first virtual behavior meets the interaction requirements, collecting activity data of the entity object in the physical environment;
[0013] The activity data is sent to the cloud server, where the activity data is used to control a first virtual object in the cloud game to perform a second virtual behavior, where the second virtual behavior is used to simulate the behavior of the physical object in the physical environment.
[0014] On the other hand, a cloud gaming-based interactive system is provided, the system comprising a display device, a control device, and a cloud server;
[0015] The control device is used to receive a first behavior control operation and send a first behavior control instruction to the cloud server, where the first behavior control instruction is used to control a first virtual object in the cloud game to perform a first virtual behavior;
[0016] The cloud server is configured to receive the first behavior control instruction, render the first virtual object based on the first behavior control instruction, and obtain a first rendering result; and feed back the first rendering result to the display device;
[0017] The display device is configured to receive the first rendering result sent by the cloud server, and display the first virtual object performing the first virtual behavior in the cloud game;
[0018] The control device is further used to collect activity data of a physical object in a physical environment when the first virtual behavior meets the interaction requirements; send the activity data to the cloud server, where the activity data is used to control the first virtual object to perform a second virtual behavior, where the second virtual behavior is used to simulate the behavior of the physical object in the physical environment;
[0019] The cloud server is configured to render the first virtual object based on the activity data to obtain a second rendering result; and feed back the second rendering result to the display device;
[0020] The display device is used to receive the second rendering result sent by the cloud server, and display the first virtual object performing the second virtual behavior in the cloud game.
[0021] On the other hand, a cloud gaming-based interactive device is provided, the device comprising:
[0022] A first display module, configured to display a first virtual object controlled by a control device in the cloud game, wherein the first virtual object performs a first virtual behavior in the cloud game, and the first virtual behavior is a behavior performed based on a control control in the control device;
[0023] The first display module is further configured to display the first virtual object performing a second virtual behavior when the first virtual behavior meets the interaction requirement;
[0024] The second virtual behavior is a behavior generated based on the activity data collected by the control device after collecting activity data of the entity object in the physical environment, and the second virtual behavior is used to simulate the behavior of the entity object in the physical environment.
[0025] On the other hand, a cloud gaming-based interactive device is provided, the device comprising:
[0026] A receiving module, configured to receive a first behavior control operation on a control device, and send a first behavior control instruction to a cloud server, wherein the first behavior control instruction is used to control a first virtual object in the cloud game to perform a first virtual behavior;
[0027] A collection module, used for collecting activity data of entity objects in the physical environment when the first virtual behavior meets the interaction requirements;
[0028] A sending module is used to send the activity data to the cloud server, where the activity data is used to control the first virtual object in the cloud game to perform a second virtual behavior, where the second virtual behavior is used to simulate the behavior of the physical object in the physical environment.
[0029] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement an interactive method based on cloud gaming as described in any of the above embodiments.
[0030] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, a code set or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the cloud gaming-based interactive method as described in any of the above embodiments.
[0031] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes any of the cloud game-based interactive methods described in the above embodiments.
[0032] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:
[0033] In the cloud gaming scenario, players can control the virtual objects displayed on the display device by operating the control device; when the virtual behavior performed by the virtual object meets the interaction requirements (for example, triggering the highlight moment in the cloud game), the control device will automatically collect the player's activity data in the current physical environment; based on the activity data, the virtual object displayed on the display device will simulate the player's behavior in the physical environment, realizing the interaction between the player and the game character in the cloud game, and enhancing the interactivity between the player and the cloud game. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0035] Figure 1 It is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;
[0036] Figure 2 is a flow chart of an interactive method based on cloud gaming provided by an exemplary embodiment of the present application;
[0037] Figure 3 is a flow chart of an interactive method based on cloud gaming provided by another exemplary embodiment of the present application;
[0038] Figure 4 is a schematic diagram of a game entry interface of a cloud gaming application provided by an exemplary embodiment of the present application;
[0039] Figure 5 is a schematic diagram of a connection guide interface provided by an exemplary embodiment of the present application;
[0040] Figure 6 is a flow chart of an interactive method based on cloud gaming provided by another exemplary embodiment of the present application;
[0041] Figure 7is a flow chart of an interactive method based on cloud gaming provided by yet another exemplary embodiment of the present application;
[0042] Figure 8 is a schematic diagram of a scanning interface provided by an exemplary embodiment of the present application;
[0043] Fig. 9 is a schematic diagram of a virtual key interface provided by an exemplary embodiment of the present application;
[0044] Fig.10 is a schematic diagram of a notification message provided by an exemplary embodiment of the present application;
[0045] Fig.11 is a flow chart of an interactive system based on cloud gaming provided by an exemplary embodiment of the present application;
[0046] Fig.12 is an overall flow chart of an interactive method based on cloud gaming provided by an exemplary embodiment of the present application;
[0047] Fig.13 is a flowchart of facial action simulation provided by an exemplary embodiment of the present application;
[0048] Fig.14 is a flow chart of a sound simulation provided by an exemplary embodiment of the present application;
[0049] Fig.15 is a structural block diagram of an interactive device based on cloud gaming provided by an exemplary embodiment of the present application;
[0050] Fig.16 is a structural block diagram of an interactive device based on cloud gaming provided by another exemplary embodiment of the present application;
[0051] Fig.17 is a structural block diagram of an interactive device based on cloud gaming provided by another exemplary embodiment of the present application;
[0052] Fig.18 It is a structural block diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solution and advantages of the present application clearer, the implementation mode of the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.
[0054] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first" and "second", nor is there any limitation on quantity and execution order.
[0055] In the related art, cloud games display the game screen through a display device, and control the game through a control device (such as a mobile phone, a game controller, etc.). When playing a game, if a player triggers a highlight moment, a special effect animation is usually displayed on the display device or a vibration control device is used to prompt the player. However, in the related art, in the scenario where the player triggers a highlight moment, the feedback given by the cloud game to the player is one-way, and the interactivity between the player and the cloud game is poor.
[0056] The present application provides an interactive method based on cloud gaming. In a cloud gaming scenario, a player can control a virtual object displayed on a display device by operating a control device. When the virtual behavior performed by the virtual object meets the interactive requirements (for example, triggering a highlight moment in a cloud game), the control device will automatically collect the player's activity data in the current physical environment. Based on the activity data, the virtual object displayed on the display device will simulate the player's behavior in the physical environment, thereby realizing the interaction between the player and the game character in the cloud game and enhancing the interactivity between the player and the cloud game.
[0057] In addition, when players trigger highlight moments, the method of capturing player behaviors and simulating them on game characters improves the effectiveness of prompting players' highlight moments, while improving the player's gaming experience, thereby increasing the player retention rate of cloud games.
[0058] Figure 1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application, such as Figure 1 As shown, the implementation environment includes: a display device 110, a control device 120 and a cloud server 130, the display device 110 and the cloud server 130 are connected via a communication network, and the control device 120 and the cloud server 130 are connected via a communication network.
[0059] The display device 110 can be implemented as a terminal device with a picture display function, such as a television, a tablet computer, a laptop computer, a mobile phone, a vehicle-mounted terminal, etc. A cloud game application is installed and run in the display device 110, and the cloud game application is used to provide a cloud game function.
[0060] Illustratively, the cloud gaming application can implement a cloud gaming application, which is a cross-terminal gaming platform for providing multiple cloud gaming portals; the cloud gaming application enables terminals to play multiple cloud games through cloud gaming portals without downloading games.
[0061] In the embodiment of the present application, the cloud game can be any one of a simulation program, a virtual reality (VR) game, an augmented reality (AR) game, a three-dimensional map game, a first-person shooter game (FPS), a third-person shooter game (TPS), a multiplayer online tactical competitive game, a strategy game (Simulation Game, SLG), etc.
[0062] The control device 120 can be implemented as a terminal device such as a smart handle, a smart keyboard, a television, a tablet computer, a laptop computer, a mobile phone, a vehicle-mounted terminal, etc. A cloud game control program is installed and run in the control device 120, and the cloud game control program is used to provide a control function for the cloud game application program.
[0063] Optionally, the cloud game control program can generate corresponding control instructions based on the operation of the physical keys received on the control device 120; the control instructions are used to control the cloud game application on the display device 110; the cloud game control program can also generate corresponding control instructions based on the operation of the virtual keys received on the control device 120; the control instructions are used to control the cloud game application on the display device 110. Among them, the virtual keys can be functions provided by the cloud game control program. Schematically, the cloud game control program provides a virtual key interface, receives the player's operation on the virtual key interface, and thus controls the cloud game application on the display device 110.
[0064] The control device 120 also needs to provide a multimedia data (e.g., motion data and sound data) acquisition function. Optionally, the control device 120 includes a video acquisition device (e.g., a camera) and an audio acquisition device (e.g., a microphone). Schematically, the control device 120 acquires motion data through the video acquisition device and acquires sound data through the audio acquisition device.
[0065] In some embodiments, the control device 120 and the display device 110 may be connected via wired or wireless communication. For example, the display device 110 may be connected to the control device 120 via Bluetooth, Wireless Fidelity (WiFi), or the like.
[0066] When the control device 120 is connected to the display device 110, the data generated on the control device 120, such as the first behavior control instruction and activity data, can be uploaded by the control device 120 to the display device 110, and the display device 110 sends the data to the cloud server 130; or, the control device 120 directly uploads the data to the cloud server 130.
[0067] In an embodiment of the present application, the display device 110 is used to display the game screen of the cloud game; the control device 120 is used to receive the game operation of the cloud game; and the cloud server 130 is used to provide rendering services for the game screen of the cloud game.
[0068] Illustratively, after receiving the first behavior control operation, the control device 120 sends the first behavior control instruction to the cloud server 130; the cloud server 130 renders the first virtual object based on the first behavior control instruction, and feeds back the first rendering result to the display device 110; after receiving the first rendering result, the display device 110 displays that the first virtual object performs the first virtual behavior in the cloud game; when the first virtual behavior meets the interaction requirements, the control device 120 collects activity data of the entity object in the physical environment, and sends the activity data to the cloud server 130; the cloud server 130 renders the first virtual object based on the activity data, and feeds back the second rendering result to the display device 110; after receiving the second rendering result sent by the cloud server, the display device 110 displays that the first virtual object performs the second virtual behavior in the cloud game, and the second virtual behavior is used to simulate the behavior of the entity object in the physical environment.
[0069] It is worth noting that the cloud server 130 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. The cloud server 130 can also provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.
[0070] Among them, cloud technology refers to a hosting technology that unifies hardware, software, network and other series of resources in a wide area network or local area network to realize data calculation, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model application, which can form a resource pool, which is used on demand and flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, each item may have its own identification mark in the future, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately, and all kinds of industry data require strong system backing support, which can only be achieved through cloud computing. Optionally, the cloud server 130 can also be implemented as a node in the blockchain system.
[0071] It should be noted that before collecting relevant data of users and during the process of collecting relevant data of users, this application can display a prompt interface, pop-up window or output voice prompt information, and the prompt interface, pop-up window or voice prompt information is used to prompt the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining relevant data of users after obtaining the confirmation operation issued by the user to the prompt interface or pop-up window, otherwise (that is, when the confirmation operation issued by the user to the prompt interface or pop-up window is not obtained), the relevant steps of obtaining relevant data of users are terminated, that is, the relevant data of users are not obtained. In other words, all user data collected by this application are collected with the consent and authorization of the user, and the collection, use and processing of relevant user data need to comply with the relevant laws, regulations and standards of relevant countries and regions.
[0072] Combined with the above introduction and implementation environment, Figure 2 This is a flow chart of a cloud gaming-based interactive method provided in an embodiment of the present application. Figure 1 Taking the display device shown in FIG. 1 as an example, the method includes:
[0073] Step 201: Display a first virtual object controlled by a control device in a cloud game.
[0074] The first virtual object performs a first virtual behavior in the cloud game, and the first virtual behavior is a behavior performed based on a control control in the control device.
[0075] The above-mentioned cloud game refers to a game mode based on cloud computing. In the operation mode of cloud games, all games are run on cloud servers, and the cloud servers transmit the rendered game screens to terminal devices (such as the display device in this application) through the network for display. Optionally, the cloud game can be any one of the games such as simulation programs, virtual reality games, augmented reality games, three-dimensional map games, first-person shooter games, third-person shooter games, multiplayer online tactical competitive games, strategy games, etc., and the embodiments of this application do not limit this.
[0076] In some embodiments, a target application is installed in the display device, and a first account is logged in to the target application; a cloud game entrance is provided in the target application, and the cloud game entrance is used to trigger a game interface of the cloud game; optionally, in response to receiving a trigger operation on the cloud game entrance in the target application, a game interface of the cloud game is displayed, and the game interface includes a first virtual object.
[0077] In the embodiment of the present application, the display device carries out the display of the game screen, and the control device carries out the game operation. Schematically, the control device receives first operation data, and the first operation data corresponds to the first virtual behavior; the control device sends the first behavior data to the cloud server, and the cloud server renders the game screen based on the first behavior data to obtain a first rendering screen, and sends the first rendering screen to the display device; the display device displays the first rendering screen, and the first rendering screen is used to instruct the first virtual object to perform the first virtual behavior.
[0078] Optionally, the display device can be any terminal device with a display screen, such as a television, a mobile phone, a laptop computer, a vehicle-mounted terminal, etc.; the control device can be any terminal device with activity data collection and control functions, such as a mobile phone, a tablet computer, etc., which is not limited in the embodiments of the present application.
[0079] Optionally, the display device and the control device may be connected via wired or wireless means, for example, the display device may be connected to the control device via Bluetooth, Wireless Fidelity (WiFi), etc.
[0080] The virtual behavior includes at least one of a virtual action and a virtual sound.
[0081] Optionally, the virtual action refers to an action performed by the first virtual object in the cloud game, and the virtual action includes body movements, such as adjusting body posture, crawling, walking, running, riding, jumping, driving, picking up, shooting, attacking, throwing, etc.; the virtual action also includes facial movements, such as smiling, blinking, raising eyebrows, frowning, etc. The virtual sound refers to the sound emitted by the first virtual object in the cloud game.
[0082] Step 202: Display the first virtual object to perform a second virtual behavior when the first virtual behavior meets the interaction requirement.
[0083] The second virtual behavior is a behavior generated based on the activity data collected by the control device after collecting activity data of the physical object in the physical environment. The second virtual behavior is used to simulate the behavior of the physical object in the physical environment.
[0084] In some embodiments, the above-mentioned interaction requirements may be requirements customized by the game system of the cloud game; or they may be requirements customized by the first account (ie, the player).
[0085] In some embodiments, when the first virtual behavior triggers a "highlight moment" in the cloud game (i.e., the player's performance in the cloud game is relatively outstanding), the first virtual object is displayed to perform the second virtual behavior. Optionally, when the first virtual behavior performed by the first virtual object triggers the first virtual object to complete a preset task in the cloud game, the first virtual object is displayed to perform the second virtual behavior.
[0086] Illustratively, when a first virtual behavior performed by a first virtual object triggers the first virtual object to defeat a preset number of enemy virtual objects, the first virtual object is displayed to perform a second virtual behavior; when the first virtual behavior performed by the first virtual object triggers the first virtual object's reward score to reach a preset score, the first virtual object is displayed to perform the second virtual behavior.
[0087] The above-mentioned physical environment refers to the real environment where the current control device is located, and the physical object is the object in the physical environment within the data collection range of the control device. In an illustrative example, the player uses the control device to perform game operations, and when the first virtual behavior meets the interaction requirements, the control device collects the corresponding activity data of the player.
[0088] The control device includes at least one of a video acquisition device and an audio acquisition device. Optionally, the activity data includes at least one of motion data and sound data of the physical object, wherein the motion data includes body motion data and facial motion data.
[0089] Optionally, the motion data is video data collected by a video acquisition device; and the sound data is audio data collected by an audio acquisition device.
[0090] Illustratively, when the control device is implemented as a smart phone and the display device is implemented as a television, the camera on the smart phone is the video acquisition device, and the microphone on the smart phone is the audio acquisition device; the TV displays a game interface of the cloud game, which includes virtual objects. Players perform game operations through smart phones to control the behaviors of virtual objects in the cloud game. When the first virtual action performed by the virtual object triggers a "highlight moment" in the cloud game (that is, the first virtual action meets the interaction requirements), the player's current action data is collected through the camera of the smart phone, and the player's current sound data is collected through the microphone of the smart phone; the smart phone sends the collected sound data and action data to the cloud server, and the cloud server renders the game screen according to the sound data and action data to obtain a second rendered screen, and sends the second rendered screen to the TV; the TV displays the second rendered screen, and the second rendered screen is used to instruct the first virtual object to perform a second virtual behavior, that is, to display the first virtual object simulating the player's actions and expressions.
[0091] In summary, the cloud gaming-based interactive method provided by the embodiments of the present application, in a cloud gaming scenario, players can control virtual objects displayed on a display device by operating a control device; when the virtual behavior performed by the virtual object meets the interactive requirements (for example, triggering a highlight moment in a cloud game), the control device will automatically collect the player's activity data in the current physical environment; based on the activity data, the virtual object displayed in the display device will simulate the player's behavior in the physical environment, thereby realizing the interaction between the player and the game character in the cloud game and enhancing the interactivity between the player and the cloud game.
[0092] In some optional embodiments, before displaying the first virtual object controlled by the control device in the cloud game, it is necessary to connect the display device and the control device. Figure 3 This is a flow chart of a cloud gaming-based interactive method provided in an embodiment of the present application. Figure 1 Taking the display device shown in FIG. 1 as an example, the method includes:
[0093] Step 301, display the device connection interface corresponding to the cloud game.
[0094] The device connection interface is used to establish a connection between the display device and the control device, that is, to connect the control device and the display device.
[0095] In some embodiments, the device connection interface is a device connection interface corresponding to a cloud gaming application. The cloud gaming application is a cross-terminal gaming platform that provides multiple cloud gaming portals; the cloud gaming application enables the terminal to play multiple cloud games through the cloud gaming portal without downloading the game.
[0096] Optionally, a first account is logged in to the cloud gaming application; before displaying the device connection interface corresponding to the cloud gaming application, the display device displays the application interface corresponding to the cloud gaming application, the application interface including multiple cloud gaming entrances, and the cloud gaming entrance is used to enter the game interface of the cloud gaming.
[0097] Schematically, multiple cloud game entrances are displayed in the application interface of the cloud game application, and the player can click on any cloud game entrance to enter the game interface of the cloud game.
[0098] Please refer to Figure 4 , which shows a schematic diagram of an application interface of a cloud gaming application. Figure 4 As shown, in interface 400, multiple different cloud game entrances are displayed. By triggering cloud game entrance 401, you can enter the game interface corresponding to cloud game 2.
[0099] Optionally, when the cloud gaming application corresponding to the first account is not connected to a control device, a device connection interface corresponding to the cloud gaming application is displayed.
[0100] In some embodiments, a cloud gaming control program is installed in the control device, and the cloud gaming control program is used to provide control functions for the cloud gaming application. The cloud gaming control program provides a virtual button interface, and the virtual button interface is used to receive operations on the cloud gaming application.
[0101] Optionally, the device connection interface includes a graphic identification code, which is used for the controlled device to connect the display device and the control device after scanning.
[0102] The graphic identification code is used to connect the control device and the display device wirelessly and trigger the display of the virtual key interface on the control device. Schematically, the graphic identification code is implemented as a QR code, which stores the page address of the virtual key interface and the socket link corresponding to the display device, so that the control device can connect the control device and the display device after scanning the QR code, and trigger the display of the virtual key interface on the control device.
[0103] Indicatively, the background of the cloud gaming application will detect whether the first account is currently connected to a control device. If the first account is not connected to a control device, a connection guidance interface (i.e., the above-mentioned device connection interface) will automatically pop up to guide the player to connect to the control device.
[0104] Please refer to Figure 5 , which shows a schematic diagram of a connection guide interface. Figure 5As shown, a QR code provided by the cloud game control program is displayed on the connection guide interface 500. When the player operates the control device to scan the QR code, if the display device and the control device are in the same network (i.e., the connected networks are the same), a connection relationship between the display device and the control device is established, and a virtual key interface is displayed on the control device. The player can control the cloud game application in the display device by triggering the virtual keys on the virtual key interface.
[0105] Step 302: When the display device and the control device are connected, display the first virtual object controlled by the control device in the cloud game.
[0106] Illustratively, when a connection is established between the display device and the control device, the control device can control the display device to enter a cloud game and display a game interface of the cloud game, where the game interface of the cloud game displays a first virtual object; after entering the cloud game interface, the control device can control the first virtual object to perform a virtual behavior on the game interface displayed by the display device, for example: the control device controls the first virtual object to perform a first virtual behavior on the game interface displayed by the display device.
[0107] The virtual behavior includes at least one of a virtual action and a virtual sound.
[0108] Optionally, the virtual action refers to an action performed by the first virtual object in the cloud game, and the virtual action includes virtual body movements, such as adjusting body posture, crawling, walking, running, riding, jumping, driving, picking up, shooting, attacking, throwing, etc.; the virtual action also includes virtual facial movements (i.e., virtual expressions), such as smiling, blinking, raising eyebrows, frowning, etc. The virtual sound refers to the sound emitted by the first virtual object in the cloud game.
[0109] Step 303: Display the first virtual object to perform a second virtual behavior when the first virtual behavior meets the interaction requirement.
[0110] Optionally, the second virtual behavior includes at least one of a virtual action and a virtual sound:
[0111] 1. When the first virtual behavior meets the interaction requirement, display the first virtual object to perform the virtual action.
[0112] Among them, the virtual action is an action generated based on the action data collected by the video acquisition device in the control device after the entity object in the physical environment is collected. The virtual action is used to simulate the action of the entity object in the physical environment.
[0113] In some embodiments, the virtual action includes a virtual limb action. Optionally, the first virtual object is displayed to perform a virtual limb action, wherein the virtual limb action is a limb action generated based on the limb action data collected by a video acquisition device in the control device after the limb action data of the physical object in the physical environment is collected, and the virtual limb action is used to simulate the limb action of the physical object in the physical environment.
[0114] In some other embodiments, the virtual action includes a virtual facial action. Optionally, the first virtual object is displayed to perform a virtual facial action, wherein the virtual facial action is a facial action generated based on facial action data collected by a video acquisition device in the control device after collecting facial action data of a physical object in a physical environment, and the virtual facial action is used to simulate the facial action of the physical object in the physical environment.
[0115] 2. When the first virtual behavior meets the interaction requirements, play the virtual sound of the first virtual object in the cloud game.
[0116] Among them, virtual sound is the sound generated based on the sound data collected by the audio collection device in the control device after collecting sound data of the entity objects in the physical environment. The virtual sound is used to simulate the sound of the entity objects in the physical environment.
[0117] Optionally, virtual sound is used to simulate the sound content of a physical object in a physical environment; or, virtual sound is used to simulate the sound timbre of a physical object in a physical environment; or, virtual sound is used to simulate the sound content and sound timbre of a physical object in a physical environment.
[0118] Illustratively, if the player's voice collected by the control device is "I'm so awesome!" and the player's timbre is that of a teenager, the voice "I'm so awesome!" can be played based on the original timbre of the first virtual object; or, the voice "I'm so awesome!" can be played by simulating a teenager's voice.
[0119] Alternatively, simulate the boy's voice to play the voice content generated by the cloud game.
[0120] Optionally, the voice content generated by the cloud game can be the content in the voice library stored in the cloud game. For example, there is a "Highlight Moment" voice library corresponding to the "Highlight Moment" triggered by the player, and a voice can be randomly selected from the "Highlight Moment" voice library for playback. The "Highlight Moment" voice library can be a voice library set by the system or a voice library set by the player.
[0121] Optionally, the voice content generated by the cloud game can also be voice content automatically generated by performing emotion recognition on the collected activity data (at least one of the action data and the sound data). Indicatively, it is assumed that the result of the emotion recognition is a pleasure score, which is used to indicate the degree of pleasure of the entity object, ranging from extreme pain to extreme ecstasy. The higher the pleasure score, the stronger the pleasure. When the pleasure score corresponding to the virtual action is recognized to be high, voice content expressing strong pleasure can be generated, such as: "I'm awesome!", "How can I be so good!", etc.; when the pleasure score corresponding to the virtual action is recognized to be low, voice content expressing slight pleasure can be generated, such as: "Keep working hard", etc.
[0122] In some embodiments, when the first virtual behavior meets the interaction requirements, text information corresponding to the virtual sound is displayed. Schematically, a bubble is displayed above the head of the first virtual object, and the text information of the virtual sound is displayed in the bubble.
[0123] The text information of the virtual sound is the sound content text corresponding to the virtual sound. For example, assuming that the virtual sound is “I am so awesome!”, the text information of the virtual sound is [I am so awesome!].
[0124] Optionally, when the first virtual behavior meets the interaction requirements, the virtual sound of the first virtual object is played in the cloud game and text information corresponding to the virtual sound is displayed; or, only the virtual sound of the first virtual object is played in the cloud game; or, only the text information corresponding to the virtual sound is displayed.
[0125] In some embodiments, when the first virtual behavior triggers a "highlight moment" in the cloud game (i.e., the player's performance in the cloud game is relatively outstanding), the first virtual object is displayed to perform the second virtual behavior. Optionally, when the first virtual behavior performed by the first virtual object triggers the first virtual object to complete a preset task in the cloud game, the first virtual object is displayed to perform the second virtual behavior.
[0126] In summary, the cloud gaming-based interactive method provided by the embodiments of the present application, in a cloud gaming scenario, players can control virtual objects displayed on a display device by operating a control device; when the virtual behavior performed by the virtual object meets the interactive requirements (for example, triggering a highlight moment in a cloud game), the control device will automatically collect the player's activity data in the current physical environment; based on the activity data, the virtual object displayed in the display device will simulate the player's behavior in the physical environment, thereby realizing the interaction between the player and the game character in the cloud game and enhancing the interactivity between the player and the cloud game.
[0127] The method provided in the embodiment of the present application, when the virtual behavior performed by the virtual object meets the interaction requirements, the virtual object displayed in the display device will simulate the player's actions in the physical environment, thereby enhancing the interactivity between the player and the display screen of the cloud game.
[0128] According to the method provided in the embodiment of the present application, when the virtual behavior performed by the virtual object meets the interaction requirements, the virtual object displayed in the display device will simulate the sound of the player in the physical environment, thereby enhancing the interactivity between the player and the playback content of the cloud game.
[0129] The method provided in the embodiment of the present application, when the virtual behavior performed by the virtual object meets the interaction requirements, the text information corresponding to the virtual sound will be displayed in the display device, further enhancing the interactivity between the player and the cloud game.
[0130] According to the method provided in the embodiment of the present application, the display device displays a device connection interface, and the graphic identification code in the device connection interface can be provided to the control device for scanning, thereby realizing a wireless connection between the display device and the control device, thereby improving the connection efficiency between the display device and the control device.
[0131] According to the method provided in the embodiment of the present application, when a first virtual behavior triggers a virtual object to complete a preset task in a cloud game, the virtual object displayed in the display device will simulate the player's behavior in the physical environment, thereby improving the effectiveness of prompting the player to complete the game task and improving the player's gaming experience, thereby improving the player retention rate of the cloud game.
[0132] Figure 6 This is a flow chart of a cloud gaming-based interactive method provided in an embodiment of the present application. Figure 1 Taking the control device shown in FIG. 1 as an example, the method includes:
[0133] Step 601: Receive a first behavior control operation on a control device, and send a first behavior control instruction to a cloud server.
[0134] The first behavior control instruction is used to control the first virtual object in the cloud game to perform a first virtual behavior.
[0135] Optionally, a cloud game control program is installed and run in the control device, and the cloud game control program is used to provide a control function for the cloud game application. Optionally, the cloud game control program is used to generate a first behavior control instruction according to the received first behavior control operation.
[0136] Optionally, the first behavior control operation is an operation received on a physical key; or, the first behavior control operation is an operation received on a virtual key.
[0137] The virtual button may be a function provided by the cloud game control program. Schematically, the cloud game control program provides a virtual button interface, and receives the player's operation on the virtual button interface, thereby controlling the cloud game application on the display device.
[0138] Optionally, after the control device generates the first behavior control instruction, it directly sends the first behavior control instruction to the cloud server; or, when the control device is connected to the display device, the control device sends the first behavior control instruction to the display device, and the display device sends the first behavior control instruction to the cloud server.
[0139] The virtual behavior includes at least one of a virtual action and a virtual sound.
[0140] Optionally, the virtual action refers to an action performed by the first virtual object in the cloud game, and the virtual action includes body movements, such as adjusting body posture, crawling, walking, running, riding, jumping, driving, picking up, shooting, attacking, throwing, etc.; the virtual action also includes facial movements, such as smiling, blinking, raising eyebrows, frowning, etc. The virtual sound refers to the sound emitted by the first virtual object in the cloud game.
[0141] Step 602 : When the first virtual behavior meets the interaction requirement, activity data of the entity object in the physical environment is collected.
[0142] The activity data is used to control the first virtual object in the cloud game to perform a second virtual behavior, and the second virtual behavior is used to simulate the behavior of the physical object in the physical environment.
[0143] In some embodiments, the above-mentioned interaction requirements may be requirements customized by the game system of the cloud game; or they may be requirements customized by the first account (ie, the player).
[0144] In some embodiments, when the first virtual behavior triggers a "highlight moment" in the cloud game (i.e., the player's performance in the cloud game is relatively outstanding), activity data collection is performed on the physical object in the physical environment. Optionally, when the first virtual behavior performed by the first virtual object triggers the first virtual object to complete a preset task in the cloud game, activity data collection is performed on the physical object in the physical environment.
[0145] In some embodiments, the control device includes at least one of a video acquisition device and an audio acquisition device. Optionally, the activity data includes at least one of motion data and sound data of the physical object, wherein the motion data includes body motion data and facial motion data.
[0146] Optionally, the motion data is video data collected by a video acquisition device; and the sound data is audio data collected by an audio acquisition device.
[0147] Optionally, the collection time for collecting activity data of entity objects in the physical environment is a time set by the background of the cloud gaming application; or, the collection time for collecting activity data of entity objects in the physical environment is a time customized by the player.
[0148] Optionally, the acquisition duration corresponding to the video acquisition device and the acquisition duration corresponding to the audio acquisition device may be the same or different.
[0149] Optionally, the collection duration of each activity data collection may be the same or different.
[0150] In some embodiments, a preset collection duration is obtained; within the preset collection duration, activity data is collected based on an alternate frame collection method.
[0151] Optionally, the motion data collection duration is obtained, and within the motion data collection duration, based on the interframe collection method, video data corresponding to the motion data is collected by a video collection device; the sound data collection duration is obtained, and within the sound data collection duration, based on the interframe collection method, audio data corresponding to the sound data is collected by an audio collection device.
[0152] In schematic form, assuming that the video capture duration is 2s, and 50 frames can be captured in 2s, then at the capture time corresponding to the first frame, the video frame is captured; at the capture time corresponding to the second frame, the video frame is not captured; at the capture time corresponding to the third frame, the video frame is captured…; assuming that the audio capture duration is 2s, and 50 frames can be captured in 2s, then at the capture time corresponding to the first frame, the audio frame is captured; at the capture time corresponding to the second frame, the audio frame is not captured; at the capture time corresponding to the third frame, the audio frame is captured….
[0153] By using the alternate frame collection method, the size of the data packet corresponding to the activity data is reduced, thereby reducing the time to send the activity data to the cloud server (i.e., reducing the network transmission time).
[0154] Step 603: Send activity data to the cloud server.
[0155] Optionally, after the control device collects the activity data, it directly sends the activity data to the cloud server; or, when the control device is connected to the display device, the control device sends the activity data to the display device, and the display device sends the activity data to the cloud server.
[0156] In summary, the cloud gaming-based interactive method provided by the embodiments of the present application, in a cloud gaming scenario, players can control virtual objects displayed on a display device by operating a control device; when the virtual behavior performed by the virtual object meets the interactive requirements (for example, triggering a highlight moment in a cloud game), the control device will automatically collect the player's activity data in the current physical environment; based on the activity data, the virtual object displayed in the display device will simulate the player's behavior in the physical environment, thereby realizing the interaction between the player and the game character in the cloud game and enhancing the interactivity between the player and the cloud game.
[0157] The method provided in the embodiment of the present application reduces the data packet size corresponding to the activity data through the method of frame-by-frame collection, thereby reducing the time for sending the activity data to the cloud server (i.e., reducing the network transmission time), and improving the efficiency of rendering the first virtual object that performs the second virtual behavior.
[0158] In some optional embodiments, before receiving the first behavior control operation on the control device, the display device and the control device need to be connected. Figure 7 This is a flow chart of a cloud gaming-based interactive method provided in an embodiment of the present application. Figure 1 Taking the display device shown in FIG. 1 as an example, the method includes:
[0159] Step 701, display the scanning interface.
[0160] Among them, the scanning interface is used to scan the graphic identification code on the display device, the graphic identification code is used to connect the display device and the control device after being scanned by the controlled device, and the graphic identification code is also used to trigger the virtual button interface corresponding to the cloud game, and the virtual button interface is used to receive control operations on the cloud game.
[0161] Optionally, the virtual button interface is an interface provided by a cloud game control program, wherein the cloud game control program is used to provide control functions for the cloud game application program.
[0162] Optionally, the graphic identification code is used to connect the control device and the display device wirelessly and trigger the display of the virtual key interface on the control device. Schematically, the graphic identification code is implemented as a QR code, which stores the page address of the virtual key interface and the socket link corresponding to the display device, so that after scanning the QR code, the control device and the display device can be connected and the virtual key interface provided by the cloud gaming application can be triggered.
[0163] For illustration, please refer to Figure 8 , which shows a schematic diagram of a scanning interface, such as Figure 8 As shown, the scanning interface 800 is used to Figure 5Scan the QR code displayed in the
[0164] Optionally, the scanning interface may be an interface provided by a cloud gaming control program; the scanning interface may also be an interface provided by other applications.
[0165] Illustratively, when the cloud game control program is implemented as a mini-program, the scanning interface may be an interface provided by a host program of the cloud game control program.
[0166] Step 702: In response to receiving a scanning operation on a graphic identification code displayed on a display device, a virtual key interface is displayed when the display device and the control device are connected.
[0167] Optionally, a virtual button interface provided by the cloud game control program is displayed.
[0168] In an illustrative manner, when the control device receives a scanning operation on the graphic identification code displayed on the display device, the display device is connected to the control device, and after the connection is successful, a virtual button interface provided by the cloud game control program is triggered to be displayed.
[0169] For illustration, please refer to Fig. 9 , which shows a schematic diagram of a virtual key interface. Fig. 9 As shown, the virtual key interface 900 includes multiple virtual keys, and by triggering these virtual keys, the first virtual object in the display device can be controlled.
[0170] Step 703: Receive a trigger operation on a virtual key on the virtual key interface, and send a first behavior control instruction to the cloud server.
[0171] Among them, the first behavior control instruction is used to control the first virtual object in the cloud game to perform a first virtual behavior.
[0172] Optionally, the trigger operation includes at least one of a single-click operation, a double-click operation, a long press operation, a sliding operation, a voice command operation, a gesture command operation, etc., which is not limited in the embodiment of the present application.
[0173] Optionally, after the control device generates the first behavior control instruction, it directly sends the first behavior control instruction to the cloud server; or, when the control device is connected to the display device, the control device sends the first behavior control instruction to the display device, and the display device sends the first behavior control instruction to the cloud server.
[0174] The virtual behavior includes at least one of a virtual action and a virtual sound.
[0175] Optionally, the virtual action refers to an action performed by the first virtual object in the cloud game, and the virtual action includes body movements, such as adjusting body posture, crawling, walking, running, riding, jumping, driving, picking up, shooting, attacking, throwing, etc.; the virtual action also includes facial movements, such as smiling, blinking, raising eyebrows, frowning, etc. The virtual sound refers to the sound emitted by the first virtual object in the cloud game.
[0176] Step 704: When the first virtual behavior meets the interaction requirement, the display device starts request information.
[0177] The device activation request information is used to request activation of the video acquisition device and the audio acquisition device on the control device.
[0178] For illustration, please refer to Fig.10 , which shows an interface diagram of a pop-up interface of a notification message (i.e., device startup request information), in which the notification message "Are you allowed to turn on the camera and microphone to record highlight moments?" is displayed in the pop-up interface 1000. When the player clicks the "OK" control 1001, it means that the player allows the camera and microphone to be turned on to record highlight moments; when the player clicks the "Cancel" control 1002, it means that the player is not authorized to turn on the camera and microphone, that is, the camera and microphone are not currently turned on, and the game rendering is performed normally afterwards.
[0179] In some embodiments, the above-mentioned interaction requirements may be requirements customized by the game system of the cloud game; or they may be requirements customized by the first account (ie, the player).
[0180] In some embodiments, when the first virtual behavior triggers a "highlight moment" in the cloud game (i.e., the player's performance in the cloud game is relatively outstanding), the device is displayed to start the request information. Optionally, when the first virtual behavior performed by the first virtual object triggers the first virtual object to complete a preset task in the cloud game, the device is displayed to start the request information.
[0181] Step 705 , in response to receiving a confirmation operation on the device start request information, collecting activity data corresponding to the entity object in the physical environment.
[0182] Optionally, the case of collecting activity data corresponding to the entity object in the physical environment includes at least one of the following cases:
[0183] 1. The control device includes a video acquisition device, and the activity data includes action data. The action data is used to control the first virtual object in the cloud game to perform a virtual action, and the virtual action is used to simulate the action of the physical object in the physical environment.
[0184] The motion data of the entity objects in the physical environment is collected, and the motion data refers to the video data collected by the video acquisition device.
[0185] Among them, the motion data includes body motion data and facial motion data.
[0186] 2. The control device includes an audio acquisition device, and the activity data includes sound data. The sound data is used to control the virtual sound of the first virtual object played in the cloud game, and the virtual sound is used to simulate the sound of the physical object in the physical environment.
[0187] Sound data is collected from physical objects in the physical environment. Sound data is the audio data collected by the audio collection device.
[0188] In some embodiments, in response to receiving a confirmation operation on the device startup request information, motion data and sound data corresponding to the physical object in the physical environment are collected.
[0189] Indicatively, Fig.10 As shown, when the player clicks the "OK" control 1001, it means that the player allows the camera and microphone to be turned on to record highlight moments, and then starts collecting the action data corresponding to the player currently within the data collection range of the camera, and the sound data corresponding to the player currently within the data collection range of the microphone.
[0190] Step 706: Send activity data to the cloud server.
[0191] Optionally, at least one of activity data, motion data and sound data is sent to a cloud server.
[0192] Optionally, after the control device collects the motion data and sound data, it directly sends the motion data and sound data to the cloud server; or, when the control device is connected to the display device, the control device sends the motion data and sound data to the display device, and the display device sends the motion data and sound data to the cloud server.
[0193] In summary, the cloud gaming-based interactive method provided by the embodiments of the present application, in a cloud gaming scenario, players can control virtual objects displayed on a display device by operating a control device; when the virtual behavior performed by the virtual object meets the interactive requirements (for example, triggering a highlight moment in a cloud game), the control device will automatically collect the player's activity data in the current physical environment; based on the activity data, the virtual object displayed in the display device will simulate the player's behavior in the physical environment, thereby realizing the interaction between the player and the game character in the cloud game and enhancing the interactivity between the player and the cloud game.
[0194] According to the method provided in the embodiment of the present application, when the virtual behavior performed by the virtual object meets the interaction requirements, the control device will collect the player's action data (for example, body movements, facial movements, etc.) in the current physical environment through the video acquisition device; based on the action data, the virtual object displayed in the display device will simulate the player's actions in the physical environment, thereby realizing the visual interaction between the player and the game character in the cloud game.
[0195] According to the method provided in the embodiment of the present application, when the virtual behavior performed by the virtual object meets the interaction requirements, the control device will collect the sound data of the player in the current physical environment through the audio collection device; based on the sound data, the virtual object displayed in the display device will simulate the sound of the player in the physical environment, thereby realizing the auditory interaction between the player and the game character in the cloud game.
[0196] According to the method provided in the embodiment of the present application, the control device connects with the display device and displays a virtual key interface by scanning the graphic identification code displayed on the display, thereby realizing a wireless connection between the display device and the control device, thereby improving the connection efficiency between the display device and the control device; at the same time, it realizes the control of the display device through the virtual key interface.
[0197] The following is an introduction to the interactive system based on cloud gaming provided in the embodiment of the application. The interactive system includes a display device, a control device and a cloud server. Fig.11 is a flow chart of an interactive system based on cloud gaming provided in an embodiment of the present application, such as Fig.11 As shown, the method comprises the following steps:
[0198] Step 1101: The control device receives a first behavior control operation and sends a first behavior control instruction to a cloud server.
[0199] The first behavior control operation is used to control the first virtual object in the cloud game to perform a first virtual behavior.
[0200] Optionally, after the control device generates the first behavior control instruction, it directly sends the first behavior control instruction to the cloud server; or, when the control device is connected to the display device, the control device sends the first behavior control instruction to the cloud gaming application in the display device, and the cloud gaming application in the display device sends the first behavior control instruction to the cloud server.
[0201] Step 1102: The cloud server receives a first behavior control instruction, and renders a first virtual object based on the first behavior control instruction to obtain a first rendering result.
[0202] Illustratively, based on the first behavior control instruction, the cloud server renders a first virtual object that executes a first virtual behavior and generates a first rendering picture, which is a first rendering result.
[0203] Step 1103: The cloud server feeds back the first rendering result to the display device.
[0204] Illustratively, the cloud server sends the first rendered image to the display device.
[0205] Step 1104: the display device receives the first rendering result sent by the cloud server, and displays the first virtual object performing the first virtual behavior in the cloud game.
[0206] Illustratively, after receiving the first rendering picture, the display device directly displays the first rendering picture, that is, displays the first virtual object performing the first virtual behavior in the cloud game.
[0207] Step 1105 : When the first virtual behavior meets the interaction requirement, the control device collects activity data of the entity object in the physical environment.
[0208] Optionally, when the first virtual behavior meets the interaction requirement, the cloud server sends an event message to the cloud gaming application in the display device, where the event message is used to indicate that the first virtual behavior meets the interaction requirement.
[0209] Subsequently, the cloud gaming application sends a notification message to the control device, which is used to notify the control device that the first virtual behavior meets the interaction requirements; after the control device receives the notification message, if there is no valid device authorization data in the control device, it displays the device start request information.
[0210] The device authorization data refers to the authorization record of the first account to the data acquisition device (eg, video acquisition device and audio acquisition device) in the control device. The device activation request information is used to request to activate the video acquisition device and audio acquisition device on the control device.
[0211] When a confirmation operation for the device start request is received, the device is controlled to collect activity data of entity objects in the physical environment.
[0212] In some embodiments, when the first virtual behavior triggers a "highlight moment" in the cloud game (i.e., the player's performance in the cloud game is relatively outstanding), activity data collection is performed on the physical object in the physical environment. Optionally, when the first virtual behavior performed by the first virtual object triggers the first virtual object to complete a preset task in the cloud game, activity data collection is performed on the physical object in the physical environment.
[0213] Optionally, the activity data includes at least one of motion data and sound data:
[0214] 1. The control device includes a video acquisition device. When the first virtual behavior meets the interaction requirements, the control device collects action data of the entity object in the physical environment through the video acquisition device.
[0215] 2. The control device includes an audio acquisition device. When the first virtual behavior meets the interaction requirements, the control device collects sound data of the entity objects in the physical environment through the video acquisition device.
[0216] Step 1106: The control device sends the activity data to the cloud server.
[0217] The activity data is used to control the first virtual object to perform a second virtual behavior, and the second virtual behavior is used to simulate the behavior of the physical object in the physical environment.
[0218] Optionally, the control device sends action data and sound data to the cloud server. The action data is used to control the first virtual object in the cloud game to perform a virtual action, and the virtual action is used to simulate the action of the physical object in the physical environment; the sound data is used to control the cloud game to play the virtual sound of the first virtual object, and the virtual sound is used to simulate the sound of the physical object in the physical environment.
[0219] Optionally, after the control device collects the motion data and sound data, it packages the motion data and sound data into a data packet, and then sends the data packet directly to the cloud server; or, when the control device is connected to the display device, the control device packages the motion data and sound data into a data packet, and then sends the data packet to the display device, and the display device sends the data packet to the cloud server.
[0220] Step 1107: The cloud server renders the first virtual object based on the activity data to obtain a second rendering result.
[0221] Optionally, based on the activity data, the cloud server renders the first virtual object that executes the second virtual behavior and generates a second rendering picture, which is a second rendering result.
[0222] Optionally, the cloud server renders the first virtual object based on the activity data to obtain a second rendering result, which includes at least one of the following situations:
[0223] 1. The cloud server performs action recognition on the video frame corresponding to the action data to obtain an action recognition result, where the action recognition result is used to indicate the action of the entity object in the physical environment; based on the action recognition result, the first virtual object is rendered to obtain a second rendering result.
[0224] Illustratively, for the action data collected by the video acquisition device in the activity data: extract the video data corresponding to the action data in the data packet, and read the duration of the video data corresponding to the action data; determine the number of video frames corresponding to the video data; traverse each video frame corresponding to the video data, and perform action recognition of the physical object on the video frame to obtain the action recognition result; the cloud server renders the virtual action of the first virtual object based on the action recognition result.
[0225] Among them, the method for performing action recognition can be implemented as follows: inputting the video frames corresponding to the video data into the model for action recognition, and outputting the action recognition results.
[0226] Schematically, a video frame corresponding to the video data is input into a face recognition model, the facial movements of the face in the current video frame are identified, and a facial movement recognition result is output; subsequently, a set of two-dimensional coordinate data of the facial movement is obtained according to the facial movement recognition result; the two-dimensional coordinate data is rendered onto a first virtual object, so that the first virtual object simulates the facial movement of the physical object.
[0227] Among them, the algorithm for facial action recognition (face recognition model) includes: at least one of: principal component analysis (PCA), support vector machine (SVM) algorithm, etc., which is not limited in the embodiment of the present application.
[0228] Among them, PCA is an unsupervised machine learning algorithm and a technology used to explore high-dimensional data structures. It is mainly used for dimensionality reduction of data. Through dimensionality reduction, features that are easier for people to understand can be discovered and the processing speed of valuable sample information can be accelerated. In addition, it can also be applied to visualization (reduced to two dimensions) and denoising.
[0229] Schematically, the video frame corresponding to the video data is input into the posture (limb movement) recognition model to identify the posture of the physical object in the current video frame, and the posture recognition result is output; subsequently, a set of two-dimensional coordinate data of the limb movement is obtained according to the posture recognition result; the two-dimensional coordinate data is rendered onto the first virtual object, so that the first virtual object simulates the posture of the physical object.
[0230] Among them, the algorithm for posture recognition (posture recognition model) includes: at least one of a key point detection method, a posture estimation method, etc., which is not limited in the embodiment of the present application.
[0231] In some embodiments, for the virtual action finally rendered on the display device, the action amplitude of the rendered virtual action can also be adjusted according to the current player's emotion.
[0232] Optionally, the action recognition result corresponds to action configuration information, and the action configuration information is used to indicate the action amplitude of the first virtual object simulating the action of the physical object in the physical environment; the cloud server performs emotion recognition on the activity data (at least one of the action data and sound data) to obtain emotion recognition results, and the emotion recognition results are used to indicate the emotions of the physical object; the action configuration information is determined based on the emotion recognition results; based on the action recognition result and the action configuration information, the first virtual object is rendered to obtain a second rendering result.
[0233] Indicatively, the emotion recognition result may be a pleasure score, which is used to indicate the degree of pleasure of the physical object, ranging from extreme pain to extreme ecstasy, and the higher the pleasure score, the stronger the pleasure. When the pleasure score corresponding to the recognized virtual action is high, the action configuration information indicates that the action amplitude is large; when the pleasure score corresponding to the recognized virtual action is low, the action configuration information indicates that the action amplitude is small.
[0234] 2. The cloud server performs speech recognition on the audio data to obtain a speech recognition result, where the speech recognition result is used to indicate the sound of the entity object in the physical environment; based on the speech recognition result, the first virtual object is rendered to obtain a second rendering result.
[0235] Optionally, the second rendering result includes voice information corresponding to the voice recognition result and text information corresponding to the voice recognition result.
[0236] Illustratively, for the sound data collected by the audio collection device in the activity data: extract the audio data corresponding to the sound data in the data packet, and read the duration of the audio data corresponding to the sound data; sample the audio data, identify the sound corresponding to the physical object therein, and obtain a sound recognition result; optionally, the cloud server can also convert the sound recognition result into text information corresponding to the sound recognition result; the cloud server renders the text information within a preset range of the first virtual object, and renders the sound recognition result on the first virtual object, so that the first virtual object simulates the words of the physical object, and displays subtitles (text information) to enhance the interactive effect.
[0237] The method for performing sound recognition may use at least one of an algorithm based on a Hidden Markov Model (HMM), an algorithm based on Dynamic Time Warping, etc., which is not limited in the embodiments of the present application.
[0238] HMM is applied to speech recognition. HMM believes that there is a "state" behind each "observation". The "state" is the "real" situation, and the observation value is the result of observing the state. For each state, there is a certain probability of observing different results, and the probability of observing different results is stored in the emission probability matrix.
[0239] Step 1108: The cloud server feeds back the second rendering result to the display device.
[0240] Illustratively, the cloud server sends the second rendered image to the display device.
[0241] Step 1109: The display device receives the second rendering result sent by the cloud server, and displays that the first virtual object performs the second virtual behavior in the cloud game.
[0242] Illustratively, after receiving the second rendering picture, the display device directly displays the second rendering picture, that is, displays the first virtual object performing the second virtual behavior in the cloud game.
[0243] In summary, the cloud gaming-based interactive method provided by the embodiment of the present application, in a cloud gaming scenario, players can control virtual objects displayed on a display device by operating a control device; when the virtual behavior performed by the virtual object meets the interactive requirements (for example, triggering a highlight moment in a cloud game), the control device will automatically collect the player's activity data in the current physical environment; after receiving the activity data, the cloud server will render and generate a rendering result; based on the rendering result, the virtual object displayed on the display device will simulate the player's behavior in the physical environment, thereby realizing the interaction between the player and the game character in the cloud game and enhancing the interactivity between the player and the cloud game.
[0244] According to the method provided in the embodiment of the present application, when the virtual behavior performed by the virtual object meets the interaction requirements, the control device will collect the player's action data (for example, body movements, facial movements, etc.) in the current physical environment through the video acquisition device; after receiving the action data, the cloud server will render and generate an action rendering result; based on the action rendering result, the virtual object displayed in the display device will simulate the player's action in the physical environment, thereby realizing the visual interaction between the player and the game character in the cloud game.
[0245] The method provided in the embodiment of the present application performs emotion recognition based on the collected activity data when performing action rendering, thereby determining the movement amplitude of the virtual object's simulated action based on the player's emotion, thereby achieving emotional sensory interaction between the player and the game character in the cloud game.
[0246] According to the method provided in the embodiment of the present application, when the virtual behavior performed by the virtual object meets the interaction requirements, the control device will collect the sound data of the player in the current physical environment through the audio collection device; after receiving the action data, the cloud server will render and generate a sound rendering result; based on the sound rendering result, the virtual object displayed in the display device will simulate the sound of the player in the physical environment, thereby realizing the auditory interaction between the player and the game character in the cloud game.
[0247] For illustration, the following example is used to illustrate the display device as a TV and the control device as a mobile phone. Fig.12 , which shows an overall flow chart of an interactive method based on cloud gaming provided in an embodiment of the present application, the method comprising:
[0248] S1201: The player starts the cloud game.
[0249] Schematically, the player launches the cloud gaming application on the TV.
[0250] S1202: The TV detects whether the mobile phone is currently connected via a socket. If not, a QR code is displayed for the player to scan.
[0251] In an illustrative manner, a player starts a cloud gaming application on a TV, and the cloud gaming application client detects whether a mobile device is currently connected via a socket. If a mobile device is currently connected via a socket, the QR code is not displayed. If not, the cloud gaming application client obtains the link of the mini-program QR code from the cloud server and displays it on the TV interface, waiting for the player to scan the link.
[0252] S1203: After the player uses the mobile phone to scan and connect to the TV, the mobile phone opens the controller mini program page.
[0253] Indicatively, after the player scans, the mobile phone opens the mini program interface and connects to the cloud gaming application client through the socket address carried in the QR code.
[0254] S1204: The player triggers a highlight moment, and the cloud server sends the event to the client, which then notifies the mini program.
[0255] Indicatively, when players play cloud games, certain highlight moments defined by the cloud game system are triggered. At this time, the cloud game server (i.e., cloud server) sends the event to the cloud game application client, and the cloud game application client notifies the mini program.
[0256] S1205: After the mini program receives the notification of the highlight moment, a pop-up window will ask the player whether to allow the camera and microphone to be turned on for recording.
[0257] Indicatively, after the mini program receives the notification, if the player has not been asked before, a pop-up window will be displayed to ask the player: A highlight moment has been triggered, do you want to allow the camera and microphone to be turned on for recording?
[0258] S1206: If the player does not allow the camera and microphone to be turned on for recording, the player proceeds with the cloud game normally.
[0259] Indicatively, when the player clicks cancel (i.e., the camera and microphone are not allowed to be turned on for recording), the applet notifies the cloud server, and the cloud server renders the game normally.
[0260] S1207: If the player allows the camera and microphone to be turned on for recording, the mini program calls the application programming interface (API) of the camera and microphone to capture the player's current actions and record the player's current voice; and assembles the data and sends it to the client, which then forwards it to the cloud server.
[0261] Indicatively, if the player clicks OK (i.e., allows the camera and microphone to be turned on for recording), the mini program notifies the cloud server and, at the same time, calls the camera and microphone APIs to turn on the camera and microphone.
[0262] Subsequently, the camera captures the current player's movements and expressions, the microphone records the current player's voice, and packages the data, which is then forwarded to the cloud server via the cloud game application client (the recording time and shooting time can be customized by the player or the cloud game).
[0263] S1208: After receiving the data packet, the cloud server parses the video in the data packet, performs face recognition and image recognition on each video frame, simulates the player's current expression and action, and renders the current simulation result on the game character.
[0264] Schematically, after receiving a data packet, the cloud server parses the data in the data packet to obtain video and sound data respectively, uses face recognition and gesture recognition on the video to simulate the current player's expression and action, and renders the simulation on the virtual game character.
[0265] S1209: The cloud server parses the audio in the data packet, translates it into text, and lets the game character play it, with text animation attached near the character's avatar.
[0266] Schematically, the cloud server parses the sound data and gives the parsed sound data to the virtual character for broadcasting. At the same time, some animations are added near the character's avatar to display the text parsed from the player's voice to enhance the fun.
[0267] S1210: When the player triggers multiple highlight moments, the above steps are repeated in sequence.
[0268] Indicatively, when a player triggers a highlight moment multiple times, if the player has agreed to record it before, there is no need for a pop-up window to be displayed later. When the player triggers a highlight moment, the highlight moment is automatically recorded and the game character is rendered and redrawn. Optionally, the highlight moment can be customized by the player and does not necessarily depend on the server.
[0269] S1211: The cloud server saves all the highlight moments of the player. After the player finishes, the cloud server makes the highlight moments into a video and pushes it to the client, which plays it.
[0270] Optionally, in the same game, the cloud server will record all the player's highlight moments and save the clips of the game character simulating the player's behavior and actions. After the game is over, the cloud server sends a notification to the cloud game application client, asking the player whether to watch his or her highlight moments. If the player chooses to watch, the video spliced with the highlight moment clips is pushed to the cloud game application client for playback.
[0271] S1212: The player ends the cloud game.
[0272] Wherein, for S1208, if PCA is used to perform face recognition on the video frame, the method for rendering the character based on the player's facial action in the cloud server may specifically include the following steps: Fig.13 Steps shown:
[0273] S1301: Receive a data packet.
[0274] That is, the data packet consisting of the facial expression data of the current player captured by the camera and the voice data of the current player recorded by the microphone is received.
[0275] S1302: Extract video data according to the tag and read the duration of the video data.
[0276] Indicatively, the video data corresponding to the expression data of the current player in the data packet is extracted according to the mark, and the duration of the video data is read.
[0277] S1303: traverse the video data, and for each video frame, use the principal component analysis method of the face recognition two-dimensional algorithm to identify the facial movements of the face in the current video frame.
[0278] S1304: According to the face recognition result, a set of two-dimensional coordinate data of facial movements is obtained, and then the two-dimensional coordinate data is rendered on the game character, and the game character is re-rendered to simulate the facial movements of the player.
[0279] Among them, for S1209, if the hidden Markov model is used to perform speech recognition on the sound data, the method for rendering the character based on the player's voice in the cloud server may specifically include the following: Fig.14 Steps shown:
[0280] S1401: Receive a data packet.
[0281] That is, the data packet consisting of the facial expression data of the current player captured by the camera and the voice data of the current player recorded by the microphone is received.
[0282] S1402: Extract sound data according to the tag and read the duration of the audio data.
[0283] Illustratively, the audio data corresponding to the sound data of the current player in the data packet is extracted according to the tag, and the duration of the audio data is read.
[0284] S1403: Sample the audio data and use the HMM speech recognition algorithm to recognize the voice of the current player.
[0285] S1404: Translate the player's voice into text, and let the game character play it, and add animation near the character to display the voice text.
[0286] Fig.15 is a structural block diagram of an interactive device based on cloud gaming provided by an exemplary embodiment of the present application, such as Fig.15 As shown, the device includes the following parts:
[0287] A first display module 1500 is used to display a first virtual object controlled by a control device in the cloud game, wherein the first virtual object performs a first virtual behavior in the cloud game, and the first virtual behavior is a behavior performed based on a control control in the control device;
[0288] The first display module 1500 is further configured to display the first virtual object performing a second virtual behavior when the first virtual behavior meets the interaction requirement;
[0289] The second virtual behavior is a behavior generated based on the activity data collected by the control device after collecting activity data of the entity object in the physical environment, and the second virtual behavior is used to simulate the behavior of the entity object in the physical environment.
[0290] In some embodiments, the second virtual behavior includes a virtual action, which is an action generated based on the action data obtained after the video capture device in the control device collects action data of the entity object in the physical environment, and the virtual action is used to simulate the action of the entity object in the physical environment; the first display module 1500 is also used to display the first virtual object performing the virtual action.
[0291] In some embodiments, the second virtual behavior includes a virtual sound, which is a sound generated based on the sound data collected by the audio collection device in the control device after collecting sound data of the physical object in the physical environment, and the virtual sound is used to simulate the sound of the physical object in the physical environment; the first display module 1500 is also used to play the virtual sound of the first virtual object in the cloud game.
[0292] In some embodiments, the first display module 1500 is further used to display text information corresponding to the virtual sound when the first virtual behavior meets the interaction requirements.
[0293] In some embodiments, the first display module 1500 is further used for:
[0294] Displaying a device connection interface corresponding to the cloud game, wherein the device connection interface includes a graphic identification code, and the graphic identification code is used to connect the display device and the control device after being scanned by the control device;
[0295] When the display device and the control device are connected, the first virtual object controlled by the control device in the cloud game is displayed.
[0296] In some embodiments, the first display module 1500 is further used to display the first virtual object performing the second virtual behavior when the first virtual behavior triggers the first virtual object to complete a preset task in the cloud game.
[0297] In summary, the cloud gaming-based interactive device provided in the embodiment of the present application allows players to control virtual objects displayed in a display device in a cloud gaming scenario by operating a control device; when the virtual behavior performed by the virtual object meets the interaction requirements (for example, triggering a highlight moment in a cloud game), the control device will automatically collect the player's activity data in the current physical environment; based on the activity data, the virtual object displayed in the display device will simulate the player's behavior in the physical environment, thereby realizing the interaction between the player and the game character in the cloud game and enhancing the interactivity between the player and the cloud game.
[0298] Fig.16is a structural block diagram of an interactive device based on cloud gaming provided by another exemplary embodiment of the present application, such as Fig.16 As shown, the device includes the following parts:
[0299] A receiving module 1600 is used to receive a first behavior control operation on a control device and send a first behavior control instruction to a cloud server, where the first behavior control instruction is used to control a first virtual object in the cloud game to perform a first virtual behavior;
[0300] A collection module 1610 is used to collect activity data of entity objects in the physical environment when the first virtual behavior meets the interaction requirements;
[0301] The sending module 1620 is used to send the activity data to the cloud server, where the activity data is used to control the first virtual object in the cloud game to perform a second virtual behavior, where the second virtual behavior is used to simulate the behavior of the physical object in the physical environment.
[0302] For illustration, please refer to Fig.17 In some embodiments, the control device includes a video acquisition device, the activity data includes action data, the action data is used to control the first virtual object in the cloud game to perform a virtual action, and the virtual action is used to simulate the action of the physical object in the physical environment; the acquisition module 1610 is also used to collect the action data of the physical object in the physical environment, and the action data is the video data collected by the video acquisition device.
[0303] In some embodiments, the control device includes an audio acquisition device, the activity data includes sound data, the sound data is used to control the virtual sound of the first virtual object played in the cloud game, and the virtual sound is used to simulate the sound of the physical object in the physical environment; the acquisition module 1610 is also used to collect the sound data of the physical object in the physical environment, and the sound data is the audio data collected by the audio acquisition device.
[0304] In some embodiments, the acquisition module 1610 is used to:
[0305] Get the preset collection duration;
[0306] Within the preset collection time, the activity data is collected based on an alternate frame collection method.
[0307] In some embodiments, the apparatus further comprises:
[0308] A second display module 1630 is used to display a scanning interface, wherein the scanning interface is used to scan a graphic identification code on the display device, wherein the graphic identification code is used to connect the display device and the control device after being scanned by the control device, and wherein the graphic identification code is also used to trigger a virtual key interface corresponding to the cloud game, wherein the virtual key interface is used to receive a control operation on the cloud game;
[0309] The second display module 1630 is further configured to display the virtual key interface in response to receiving a scanning operation on the graphic identification code displayed on the display device when the display device and the control device are connected.
[0310] In summary, the cloud gaming-based interactive device provided in the embodiment of the present application allows players to control virtual objects displayed in a display device in a cloud gaming scenario by operating a control device; when the virtual behavior performed by the virtual object meets the interaction requirements (for example, triggering a highlight moment in a cloud game), the control device will automatically collect the player's activity data in the current physical environment; based on the activity data, the virtual object displayed in the display device will simulate the player's behavior in the physical environment, thereby realizing the interaction between the player and the game character in the cloud game and enhancing the interactivity between the player and the cloud game.
[0311] It should be noted that the interactive device based on cloud gaming provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the interactive device based on cloud gaming provided in the above embodiment and the interactive method embodiment based on cloud gaming belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0312] Fig.18 The block diagram of the structure of a computer device 1800 provided by an exemplary embodiment of the present application is shown. The computer device 1800 may be a portable mobile terminal, such as a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The computer device 1800 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.
[0313] Typically, the computer device 1800 includes a processor 1801 and a memory 1802 .
[0314] The processor 1801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
[0315] The memory 1802 may include one or more computer-readable storage media, which may be non-transitory. The memory 1802 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1802 is used to store at least one instruction, which is used to be executed by the processor 1801 to implement the interactive method based on cloud gaming provided in the method embodiment of the present application.
[0316] In some embodiments, the computer device 1800 may further include other components, which can be understood by those skilled in the art. Fig.18 The structure shown in the figure does not constitute a limitation on the computer device 1800, and the computer device 1800 may include more or less components than shown in the figure, or combine some components, or adopt a different arrangement of components.
[0317] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which can be a computer-readable storage medium contained in the memory in the above embodiments; or it can be a computer-readable storage medium that exists independently and is not assembled in the terminal. The computer-readable storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement any of the cloud game-based interactive methods described in the above embodiments.
[0318] Optionally, the computer readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid state drive (SSD), or an optical disk. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM). The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.
[0319] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0320] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. An interactive method based on cloud gaming, characterized in that: The method is performed by a display device and includes: Displaying a first virtual object controlled by a control device in the cloud game, wherein the first virtual object performs a first virtual behavior in the cloud game, and the first virtual behavior is a behavior performed based on a control control in the control device; When the first virtual behavior meets the interaction requirement, displaying the first virtual object to perform a second virtual behavior; The second virtual behavior is a behavior generated based on the activity data collected by the control device after collecting activity data of the entity object in the physical environment, and the second virtual behavior is used to simulate the behavior of the entity object in the physical environment.
2. The method according to claim 1, characterized in that: The second virtual behavior includes a virtual action, wherein the virtual action is generated based on the action data collected by the video acquisition device in the control device after the action data of the entity object in the physical environment is collected, and the virtual action is used to simulate the action of the entity object in the physical environment; The displaying the first virtual object to perform a second virtual behavior includes: The first virtual object is displayed to perform the virtual action.
3. The method according to claim 1, characterized in that The second virtual behavior includes a virtual sound, wherein the virtual sound is generated based on the sound data collected by the audio collection device in the control device after collecting sound data of the physical object in the physical environment, and the virtual sound is used to simulate the sound of the physical object in the physical environment; The displaying the first virtual object to perform a second virtual behavior includes: The virtual sound of the first virtual object is played in the cloud game.
4. The method according to claim 3, characterized in that The method further comprises: When the first virtual behavior meets the interaction requirement, text information corresponding to the virtual sound is displayed.
5. The method according to any one of claims 1 to 4, characterized in that: The displaying of the first virtual object controlled by the control device in the cloud game includes: Displaying a device connection interface corresponding to the cloud game, wherein the device connection interface includes a graphic identification code, and the graphic identification code is used to connect the display device and the control device after being scanned by the control device; When the display device and the control device are connected, the first virtual object controlled by the control device in the cloud game is displayed.
6. The method according to any one of claims 1 to 4, characterized in that: The step of displaying the first virtual object to perform a second virtual behavior when the first virtual behavior meets the interaction requirement includes: When the first virtual behavior triggers the first virtual object to complete a preset task in the cloud game, the first virtual object is displayed to perform the second virtual behavior.
7. An interactive method based on cloud gaming, characterized in that: Executed by a control device, the method includes: Receiving a first behavior control operation on the control device, and sending a first behavior control instruction to the cloud server, where the first behavior control instruction is used to control a first virtual object in the cloud game to perform a first virtual behavior; When the first virtual behavior meets the interaction requirements, collecting activity data of the entity object in the physical environment; The activity data is sent to the cloud server, where the activity data is used to control a first virtual object in the cloud game to perform a second virtual behavior, where the second virtual behavior is used to simulate the behavior of the physical object in the physical environment.
8. The method according to claim 7, characterized in that The control device includes a video acquisition device, the activity data includes action data, and the action data is used to control the first virtual object in the cloud game to perform a virtual action, and the virtual action is used to simulate the action of the physical object in the physical environment; The collecting of activity data of entity objects in the physical environment includes: The action data is collected for the entity object in the physical environment, and the action data is the video data collected by the video collection device.
9. The method according to claim 7, characterized in that: The control device includes an audio acquisition device, the activity data includes sound data, the sound data is used to control the playing of the virtual sound of the first virtual object in the cloud game, and the virtual sound is used to simulate the sound of the physical object in the physical environment; The collecting of activity data of entity objects in the physical environment includes: The sound data is collected from the entity object in the physical environment, and the sound data is the audio data collected by the audio collection device.
10. The method according to any one of claims 7 to 9, characterized in that: The collecting of activity data of entity objects in the physical environment includes: Get the preset collection duration; Within the preset collection time, the activity data is collected based on an alternate frame collection method.
11. The method according to any one of claims 7 to 9, characterized in that: Before receiving the first behavior control operation on the control device and sending the first behavior control instruction to the cloud server, the method further includes: Display a scanning interface, the scanning interface is used to scan the graphic identification code on the display device, the graphic identification code is used to connect the display device and the control device after being scanned by the control device, the graphic identification code is also used to trigger the virtual key interface corresponding to the cloud game, and the virtual key interface is used to receive the control operation of the cloud game; In response to receiving a scanning operation of the graphic identification code displayed on the display device, the virtual key interface is displayed when the display device and the control device are connected.
12. An interactive system based on cloud gaming, characterized in that: The system includes a display device, a control device and a cloud server; The control device is used to receive a first behavior control operation and send a first behavior control instruction to the cloud server, where the first behavior control instruction is used to control a first virtual object in the cloud game to perform a first virtual behavior; The cloud server is configured to receive the first behavior control instruction, and render the first virtual object based on the first behavior control instruction to obtain a first rendering result; Feedback the first rendering result to the display device; The display device is configured to receive the first rendering result sent by the cloud server, and display the first virtual object performing the first virtual behavior in the cloud game; The control device is further used to collect activity data of a physical object in a physical environment when the first virtual behavior meets the interaction requirements; send the activity data to the cloud server, where the activity data is used to control the first virtual object to perform a second virtual behavior, where the second virtual behavior is used to simulate the behavior of the physical object in the physical environment; The cloud server is configured to render the first virtual object based on the activity data to obtain a second rendering result; Feedback the second rendering result to the display device; The display device is used to receive the second rendering result sent by the cloud server, and display the first virtual object performing the second virtual behavior in the cloud game.
13. The system according to claim 12, characterized in that The control device includes a video acquisition device, and the activity data includes motion data; The control device is further used for collecting the action data of the entity object in the physical environment through the video acquisition device when the first virtual behavior meets the interaction requirements; and sending the action data to the cloud server; The cloud server is also used to perform action recognition on the video frames corresponding to the action data to obtain an action recognition result, and the action recognition result is used to indicate the action of the entity object in the physical environment; based on the action recognition result, the first virtual object is rendered to obtain the second rendering result.
14. The system according to claim 13, characterized in that The action recognition result corresponds to action configuration information, and the action configuration information is used to indicate the action amplitude of the first virtual object simulating the action of the physical object in the physical environment; The cloud server is further used to perform emotion recognition on the activity data to obtain an emotion recognition result, wherein the emotion recognition result is used to indicate the emotion of the entity object; Determining the action configuration information based on the emotion recognition result; Based on the action recognition result and the action configuration information, the first virtual object is rendered to obtain the second rendering result.
15. The system according to claim 13, characterized in that The control device includes an audio acquisition device, and the activity data includes sound data; The control device is further used to collect audio data corresponding to the sound data from a physical object in a physical environment through the audio collection device when the first virtual behavior meets the interaction requirements; and send the audio data to the cloud server; The cloud server is further used to perform speech recognition on the audio data to obtain a speech recognition result, wherein the speech recognition result is used to indicate the sound of the physical object in the physical environment; Based on the speech recognition result, the first virtual object is rendered to obtain the second rendering result.
16. An interactive device based on cloud gaming, characterized in that: The device comprises: A first display module, configured to display a first virtual object controlled by a control device in the cloud game, wherein the first virtual object performs a first virtual behavior in the cloud game, and the first virtual behavior is a behavior performed based on a control control in the control device; The first display module is further configured to display the first virtual object performing a second virtual behavior when the first virtual behavior meets the interaction requirement; The second virtual behavior is a behavior generated based on the activity data collected by the control device after collecting activity data of the entity object in the physical environment, and the second virtual behavior is used to simulate the behavior of the entity object in the physical environment.
17. An interactive device based on cloud gaming, characterized in that: The device comprises: A receiving module, configured to receive a first behavior control operation on a control device, and send a first behavior control instruction to a cloud server, wherein the first behavior control instruction is used to control a first virtual object in the cloud game to perform a first virtual behavior; A collection module, used for collecting activity data of entity objects in the physical environment when the first virtual behavior meets the interaction requirements; A sending module is used to send the activity data to the cloud server, where the activity data is used to control the first virtual object in the cloud game to perform a second virtual behavior, where the second virtual behavior is used to simulate the behavior of the physical object in the physical environment.
18. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the cloud gaming-based interactive method as described in any one of claims 1 to 11.
19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor to implement the cloud gaming-based interactive method as described in any one of claims 1 to 11.
20. A computer program product, characterized in that It includes a computer program, which, when executed by a processor, implements the cloud game-based interactive method as described in any one of claims 1 to 11.
