Data processing method and related device

Abstract

The invention discloses a data processing method and a related device, which can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, auxiliary driving and the like. When a plurality of terminal devices are required to participate in the interaction scene of the interaction application together, a first data connection with a cloud server can be established based on the first terminal device. The second terminal device directly establishes a second data connection with the first terminal device, the first terminal device serves as a data transfer role between the cloud server of the interaction scene and other terminal devices, and the interaction video stream of the interaction scene provided by the cloud server is forwarded to the other terminal devices participating in the interaction scene. And the interaction information generated by the other terminal devices for the interaction scene is forwarded to the cloud server, so that the cloud server only needs to send the interaction video stream to the first terminal device instead of sending the interaction video stream to the other terminal devices, and the bandwidth pressure of the cloud server is greatly reduced.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a data processing method and related devices. Background Art

[0002] Through the application of cloud technology, the complex data processing part that was originally required to be handled by terminal devices can be moved to cloud servers with powerful computing power. Terminal devices only need to be responsible for content display and control input, reducing the resource burden of terminal devices running interactive applications.

[0003] When running an interactive application, the terminal device needs to obtain the current running screen of the interactive application from the cloud server for display. For interactive applications involving multiple people, the cloud server needs to transmit the current running screen to multiple terminal devices separately, which puts a lot of pressure on the bandwidth of the cloud server.

[0004] How to reduce the bandwidth pressure of cloud servers is an urgent problem that needs to be solved. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a data processing method and related devices, which only need to send an interactive video stream to a first terminal device without sending an interactive video stream to other terminal devices, thereby greatly reducing the bandwidth pressure of the cloud server.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] On the one hand, an embodiment of the present application provides a data processing method, which is executed by a first terminal device, and the method includes:

[0008] Establishing a first data connection with the cloud server by running an interactive application;

[0009] In response to a connection request from a second terminal device, a second data connection is established with the second terminal device, and the second terminal device and the first terminal device jointly participate in an interactive scenario of the interactive application;

[0010] The interactive video stream obtained from the first data connection is forwarded to the second terminal device via the second data connection, and the interactive information obtained from the second data connection is forwarded to the cloud server via the first data connection, wherein the interactive video stream is used to display the interactive scene, and the interactive information is used to characterize the interactive behavior performed on the interactive scene by the second terminal device.

[0011] On the other hand, an embodiment of the present application provides a data processing method, which is executed by a cloud server and includes:

[0012] In response to the first terminal device running the interactive application, establishing a first data connection with the first terminal device, and generating an interactive scene of the interactive application, wherein the interactive scene includes a plurality of interactive objects;

[0013] Acquire interaction information sent by the first terminal device through the first data connection;

[0014] Determine a target object corresponding to the interaction information from the multiple interaction objects;

[0015] The target object is controlled to perform corresponding interactive behavior in the interactive scene according to the interactive information, and an interactive video stream corresponding to the interactive scene is sent to the first terminal device through the first data connection.

[0016] On the other hand, an embodiment of the present application provides a first terminal device, wherein the first terminal device includes an establishing unit and a forwarding unit:

[0017] The establishing unit is used to establish a first data connection with the cloud server by running an interactive application;

[0018] The establishing unit is further configured to establish a second data connection with the second terminal device in response to a connection request of the second terminal device, wherein the second terminal device and the first terminal device jointly participate in the interactive scenario of the interactive application;

[0019] The forwarding unit is used to forward the interactive video stream obtained from the first data connection to the second terminal device through the second data connection, and forward the interactive information obtained from the second data connection to the cloud server through the first data connection, the interactive video stream is used to display the interactive scene, and the interactive information is used to characterize the interactive behavior based on the second terminal device on the interactive scene.

[0020] On the other hand, an embodiment of the present application provides a cloud server, wherein the cloud server includes an establishing unit, an acquiring unit, a determining unit, and a sending unit:

[0021] The establishing unit is configured to establish a first data connection with the first terminal device in response to the first terminal device running the interactive application, and generate an interactive scene of the interactive application, wherein the interactive scene includes a plurality of interactive objects;

[0022] The acquiring unit is configured to acquire the interaction information sent by the first terminal device through the first data connection;

[0023] The determining unit is used to determine a target object corresponding to the interaction information from the multiple interaction objects;

[0024] The sending unit is used to control the target object to perform corresponding interactive behavior in the interactive scene according to the interactive information, and send the interactive video stream corresponding to the interactive scene to the first terminal device through the first data connection.

[0025] On the other hand, an embodiment of the present application provides a data processing system, the system comprising a first terminal device and a cloud server:

[0026] The first terminal device is used to execute the relevant method described in the above aspects;

[0027] The cloud server is used to execute the relevant methods described in the above aspects.

[0028] In another aspect, an embodiment of the present application provides a computer device, the computer device comprising a processor and a memory:

[0029] The memory is used to store a computer program and transmit the computer program to the processor;

[0030] The processor is configured to execute the method described above according to the computer program.

[0031] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a computer device, it implements the method described in the above aspects.

[0032] On the other hand, an embodiment of the present application provides a computer program product including a computer program, which, when executed on a computer device, enables the computer device to execute the method described in the above aspects.

[0033] It can be seen from the above technical solution that for interactive applications based on cloud servers, when multiple terminal devices are required to jointly participate in the interactive scene of the interactive application, a first data connection with the cloud server can be established based on the first terminal device among the multiple terminal devices by running the interactive application. When the second terminal device among the multiple terminal devices needs to join the interactive scene, the second terminal device directly establishes a second data connection with the first terminal device, and the first terminal device acts as the data transfer role between the cloud server and other terminal devices in the interactive scene. Based on the established first data connection and second data connection, the first terminal device forwards the interactive video stream of the interactive scene provided by the cloud server to other terminal devices participating in the interactive scene, and forwards the interactive information generated by other terminal devices for the interactive scene to the cloud server, so that the cloud server only connects to one terminal device when providing processing services for the interactive scene, that is, it only needs to send the interactive video stream to the first terminal device, and does not need to send the interactive video stream to other terminal devices, which greatly reduces the bandwidth pressure of the cloud server. Moreover, since the interactive video stream can reach the second terminal device through only one layer of real-time forwarding of the first terminal device, the interactive scene displayed based on the interactive video stream on the second terminal device can also achieve good synchronization and real-time performance with the first terminal device, ensuring the user's interactive experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A schematic diagram of a data processing scenario provided in an embodiment of the present application;

[0036] Figure 2 A method flow chart of a data processing method provided in an embodiment of the present application;

[0037] Figure 3a One of the schematic diagrams of selecting interaction information for forwarding provided in an embodiment of the present application;

[0038] Figure 3b A second schematic diagram of selecting interaction information for forwarding provided in an embodiment of the present application;

[0039] Figure 4a One of the schematic diagrams for publishing the connection address information of a master device provided in an embodiment of the present application;

[0040] Figure 4bA second schematic diagram of publishing connection address information of a master device provided in an embodiment of the present application;

[0041] Figure 5 A schematic diagram of a data processing system configured with a virtual input component pair provided in an embodiment of the present application;

[0042] Figure 6 A method flow chart of another data processing method provided in an embodiment of the present application;

[0043] Figure 7a One of the schematic diagrams of terminal device transfer based on a virtual input component pair provided in an embodiment of the present application;

[0044] Figure 7b A second schematic diagram of a terminal device transfer based on a virtual input component pair provided in an embodiment of the present application;

[0045] Figure 8 A device structure diagram of a first terminal device provided in an embodiment of the present application;

[0046] Fig. 9 A device structure diagram of a cloud server provided in an embodiment of the present application;

[0047] Fig.10 A structural diagram of a data processing system provided in an embodiment of the present application;

[0048] Fig.11 A structural diagram of a terminal device provided in an embodiment of the present application;

[0049] Fig.12 A structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0051] In interactive applications implemented based on cloud technology, when the generated interactive scene involves terminal devices of multiple users, the cloud server needs to transmit the running screen of the current interactive scene to multiple terminal devices separately to ensure that the interactive scenes displayed by all terminal devices participating in the interactive scene are synchronized. This transmission method puts a lot of pressure on the bandwidth of the cloud server.

[0052] To this end, an embodiment of the present application provides a data processing method that only needs to send an interactive video stream to a first terminal device, without sending an interactive video stream to other terminal devices, thereby greatly reducing the bandwidth pressure of the cloud server.

[0053] The data processing method provided in the embodiments of the present application can be implemented by a computer device, which can be a terminal device or a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0054] Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc. The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0055] The data processing method in the embodiment of the present application involves cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to realize the calculation, storage, processing, and sharing of data. It is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the application of cloud computing business model. It can form a resource pool, which is used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites, and more portal websites. With the high development and application of the Internet industry, each item may have its own identification mark in the future, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately, and all kinds of industry data need strong system backing support, which can only be achieved through cloud computing.

[0056] The interactive applications mentioned in the embodiments of this application may involve various interactive products based on cloud technology, such as cloud games, cloud education, and cloud conferences. Among them:

[0057] Cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing technology. Cloud gaming technology enables thin clients with relatively limited graphics processing and data computing capabilities to run high-quality games. In a cloud gaming scenario, the game is not played on the player's game terminal, but on a cloud server, which renders the game scene into a video and audio stream and transmits it to the player's game terminal over the network. The player's game terminal does not need to have powerful graphics computing and data processing capabilities, but only needs to have basic streaming media playback capabilities and the ability to obtain player input commands and send them to the cloud server.

[0058] Cloud Computing Education (CCEDU for short) refers to an education platform service based on the cloud computing business model. On the cloud platform, all educational institutions, training institutions, enrollment service institutions, publicity institutions, industry associations, management institutions, industry media, legal structures, etc. are integrated into a resource pool. Each resource is displayed and interacted with each other, and communication is carried out on demand to reach an agreement, thereby reducing education costs and improving efficiency.

[0059] Cloud conferencing is an efficient, convenient and low-cost conferencing format based on cloud computing technology. Users only need to use the Internet interface and perform simple and easy-to-use operations to quickly and efficiently share voice, data files and videos with teams and customers around the world. The cloud conferencing service provider helps users operate complex technologies such as data transmission and processing during the meeting.

[0060] It is understandable that in the specific implementation of the present application, the interactive information obtained may include data related to user information, etc. When the above embodiments of the present application are applied to specific products or technologies, any one of them needs to obtain user permission or consent separately, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0061] Figure 1 A schematic diagram of a data processing system provided in an embodiment of the present application, in which a first terminal device 100, a second terminal device 200 and a cloud server 300 are included. As an example only, Figure 1 Two second terminal devices 200 are shown in the scenario, but this is not a limitation on the quantity. In actual application scenarios, the number of second terminal devices 200 can be one or more.

[0062] The first terminal device 100 and the second terminal device 200 are both terminal devices among multiple terminal devices that jointly participate in the same interactive scene. The difference lies in that the first terminal device 100 is a terminal device that establishes a data connection with the cloud server 300, and the second terminal device 200 is a terminal device that establishes a data connection with the first terminal device 100. The second terminal device 200 does not need to establish a data connection with the cloud server 300 when participating in the interactive scene.

[0063] The first terminal device 100 and the second terminal device 200 may respectively be installed with clients of the interactive application.

[0064] The interactive scenario in which the first terminal device 100 and the second terminal device 200 need to jointly participate in an interactive application may be, for example, a multiplayer online game scenario, and the interactive application provides business processing services through the cloud server 300.

[0065] The first terminal device 100 can establish a first data connection with the cloud server 300 by running the interactive application. When the second terminal device 200 needs to join the interactive scene, the second terminal device 200 directly establishes a second data connection with the first terminal device 100, and the first terminal device 100 serves as the cloud server 300 and other terminal devices (such as Figure 1 The data transfer role between two second terminal devices 200).

[0066] Based on the established first data connection and second data connection, the first terminal device 100 forwards the interactive video stream of the interactive scene provided by the cloud server 300 to the second terminal device 200, and forwards the interactive information generated by the second terminal device 200 for the interactive scene to the cloud server 300, so that when the cloud server 300 provides business processing services for the interactive scene, it only connects to one terminal device, namely the first terminal device 100. In other words, the cloud server 300 only needs to send the interactive video stream to the first terminal device 100, and does not need to send the interactive video stream to the second terminal device 200, which greatly reduces the bandwidth pressure of the cloud server. Figure 1 In the scenario shown, in the related art, the cloud server 300 needs to establish data connections with the first terminal device 100 and the second terminal device 200, and the generated interactive video stream also needs to be sent to the three terminal devices synchronously. In the present application, it is only necessary to send the interactive video stream to the first terminal device 100. In terms of transmitting the interactive video stream, the bandwidth consumption is reduced to one third of the original.

[0067] Since the interactive video stream can reach the second terminal device 200 through only one layer of real-time forwarding of the first terminal device 100, the interactive scene displayed based on the interactive video stream on the second terminal device 200 can also achieve good synchronization and real-time performance with the first terminal device 100, ensuring the user's interactive experience.

[0068] Figure 2 A method flow chart of a data processing method provided in an embodiment of the present application, the method is executed by a first terminal device, and the method includes:

[0069] S201: The first terminal device establishes a first data connection with the cloud server by running an interactive application.

[0070] As mentioned above, in the application scenario of the embodiment of the present application, at least a cloud server, a first terminal device and at least one second terminal device are included.

[0071] The cloud server is used to provide business processing for the interactive application and return the processing results to the terminal device using the interactive application.

[0072] Different users use the interactive application through a first terminal device and a second terminal device, wherein the first terminal device can be regarded as a terminal device for creating an interactive scene, such as a terminal device for starting a game, and the second terminal device can be regarded as a terminal device for joining the interactive scene, such as a terminal device for joining the game.

[0073] Interactive applications may involve various interactive products based on cloud technology, such as cloud games, cloud education, and cloud conferences. When the interactive application is a cloud game, the interactive scenario of the interactive application may be a multiplayer game; when the interactive application is cloud education, the interactive scenario of the interactive application may be an online class; when the interactive application is a cloud conference, the interactive scenario of the interactive application may be an online conference.

[0074] Since the business processing when the interactive application is running is completed by the cloud server, when the first terminal device runs the interactive application to prepare to create the interactive scene, a first data connection with the cloud server can be established.

[0075] The first data connection is used for data transmission between the first terminal device and the cloud server. For example, the first terminal device can obtain an interactive video stream for displaying an interactive scene from the cloud server through the first data connection. The first terminal device can also send its own interactive information to the cloud server through the first data connection and forward the interactive information obtained from the second terminal device.

[0076] After the first terminal device establishes a first data connection with the cloud server based on the interaction scenario, other terminal devices requesting to join the interaction scenario will not establish a data connection with the cloud server, but will establish a second data connection with the first terminal device.

[0077] In the embodiment of the present application, the first terminal device will act as a transfer device between the cloud server and the second terminal device. The specific control logic will be described in detail later.

[0078] S202: In response to a connection request from a second terminal device, the first terminal device establishes a second data connection with the second terminal device.

[0079] As mentioned above, the second terminal device and the first terminal device jointly participate in the interactive scenario of the interactive application. The number of second terminal devices is not limited in this application and can be at least one.

[0080] After the first terminal device establishes a first data connection with the cloud server, all second terminal devices that subsequently request to join the interactive scene can establish a one-to-one corresponding second data connection with the first terminal device, that is, different second terminal devices establish different second data connections with the first terminal device. For example, there are three second terminal devices, terminal device 1, terminal device 2, and terminal device 3. Among these three second terminal devices, terminal device 1 establishes a second data connection 1 with the first terminal device, terminal device 2 establishes a second data connection 2 with the first terminal device, and terminal device 3 establishes a second data connection 3 with the first terminal device.

[0081] The second data connection is used for data transmission between the first terminal device and the second terminal device. For example, the first terminal device can forward the interactive video stream obtained from the cloud server to the second terminal device via the second data connection, and the first terminal device can also obtain interactive information for the interactive scene from the second terminal device via the second data connection.

[0082] S203: The first terminal device forwards the interactive video stream obtained from the first data connection to the second terminal device through the second data connection, and forwards the interactive information obtained from the second data connection to the cloud server through the first data connection.

[0083] The interactive video stream is used to display the interactive scene. When the terminal device obtains the interactive video stream, it can display the interactive scene corresponding to the interactive video stream on the display interface of the terminal device through decoding. Before the interactive scene is closed, the content in the interactive scene will change based on the interactive information sent by the first terminal device and the second terminal device. Therefore, the interactive video stream is sent from the cloud server to the first terminal device in real time based on the change of the interactive scene. Correspondingly, the first terminal device will also forward the obtained interactive video to the second terminal device in real time.

[0084] The interaction information is used to characterize the interaction behavior of the second terminal device on the interaction scene. The interaction behavior may include controlling the corresponding object in the interaction scene, such as controlling the player character to release skills, etc., or may include interaction behavior on the interaction scene itself, such as input text, voice, etc., or may include file sharing, editing of content in the interaction scene, etc.

[0085] After establishing a first data connection with the cloud server and a second data connection with the second terminal device, the first terminal device has the basis to serve as a transfer device between the cloud server and the second terminal device.

[0086] Through the first data connection, the first terminal device can forward the interaction information obtained from the second terminal device to the cloud server, or send the interaction information generated by itself to the cloud server, so that the cloud server can respond to the obtained interaction information in real time to update the interaction scene. At the same time, through the first data connection, the first terminal device can also obtain the interaction video stream from the cloud server, and display the interaction scene on the first terminal device by decoding and rendering the interaction video stream locally.

[0087] Through the second data connection, the first terminal device can forward the interactive video stream obtained from the cloud server to the second terminal device, so that the second terminal device can decode and render the interactive video stream locally to display the interactive scene on the second terminal device. At the same time, through the second data connection, the first terminal device can also obtain interactive information used to characterize the interactive behavior of the second terminal device on the interactive scene, and the interactive information will be forwarded to the cloud server by the first terminal device in real time through the first data connection.

[0088] It can be seen that when the cloud server provides processing services for the interactive scenario, it only needs to connect to one terminal device. That is, it only needs to send the interactive video stream to the first terminal device, and there is no need to send the interactive video stream to other terminal devices, which greatly reduces the bandwidth pressure of the cloud server.

[0089] Since the interactive video stream can reach the second terminal device through only one layer of real-time forwarding of the first terminal device, the interactive scene displayed based on the interactive video stream on the second terminal device can also achieve good synchronization and real-time performance with the first terminal device, ensuring the user's interactive experience.

[0090] By using the first terminal device as a transit device between the cloud server and the second terminal device, the cloud server only needs to connect to one terminal device, namely the first terminal device, when providing business processing services for interactive scenarios involving multiple people. Therefore, the cloud server only needs to run one application instance, such as a game instance, to carry the service, which effectively reduces the resource maintenance pressure of the cloud server and improves processing efficiency.

[0091] For interaction scenarios involving multiple participants, different interaction scenarios may have different interaction requirements or rules.

[0092] Therefore, in a possible implementation manner, forwarding the interaction information obtained from the second data connection to the cloud server through the first data connection in S203 includes:

[0093] S2031: In response to acquiring the interaction information from the second data connection, determining interaction permission information for the interaction scenario;

[0094] S2032: In response to determining that the second terminal device has the interaction authority identified by the interaction authority information, forward the interaction information to the cloud server through the first data connection.

[0095] That is, before forwarding the interaction information of the second terminal device, the first terminal device needs to determine whether the second terminal device satisfies the current interaction authority information.

[0096] The interaction permission information is used to characterize the rules and permissions for interactive behaviors in the current interactive scenario, such as which types of interactive behaviors are allowed, which types of interactive behaviors are prohibited, which accounts (corresponding to terminal devices) have the permission to perform interactive behaviors, and which accounts do not have the permission to perform interactive behaviors.

[0097] It can be seen that by introducing interaction permission information, the embodiments of the present application can be effectively applied to interaction scenarios with different interaction requirements, improve the forwarding accuracy when the first terminal device acts as a transit device, and the management of interaction permission information does not require the intervention of the cloud server, thereby reducing the processing complexity of the cloud server.

[0098] In a possible implementation, the method further includes:

[0099] S2033: In response to determining that the second terminal device does not have the interaction authority identified by the interaction authority information, prohibiting forwarding the interaction information to the cloud server.

[0100] As mentioned above, by introducing interaction permission information, the first terminal device can reasonably control the interaction behavior in the interaction scene, avoid the interaction behavior that does not meet the interaction requirements of the current interaction scene from being responded to by the cloud server, and reduce the interference of improper interaction behavior to the interaction scene, thereby maintaining the interaction quality of the interaction scene.

[0101] It should be noted that the interaction permission information may be fixed in the interaction scenario or may be dynamically adjusted in the interaction scenario, and this application does not limit this.

[0102] For example Figure 3a As shown, when the interactive scene is an online class, students are not allowed to speak or talk during the teacher's lecture. At this time, the interactive permission information is used to indicate that the first terminal device 100 (assuming that it is the teacher's) has the permission to implement the interactive behavior, and the second terminal device 200 (assuming that it is the student's) does not have the permission to implement the interactive behavior. When students need to answer questions after teaching a section of knowledge, the teacher can adjust the interactive permission information in real time to indicate that both the first terminal device 100 and the second terminal device 200 have the permission to implement the interactive behavior.

[0103] In this example, if the first terminal device 100 obtains interaction information such as interaction information 1 and interaction information 2 sent by the second terminal device 200 during the question-answering period, it can be determined through the interaction permission information at this time that the second terminal device 200 has the authority to perform interactive behavior, and the first terminal device 100 will forward the interaction information 1 and interaction information 2 to the cloud server 300 (not shown in the figure).

[0104] If the first terminal device 100 obtains interactive information such as interactive information 1 and interactive information 2 sent by the second terminal device 200 during a lecture, it can be determined through the interactive permission information at this time that the second terminal device 200 does not have the permission to perform interactive behavior. The first terminal device 100 will not forward the interactive information 1 and interactive information 2 to the cloud server 300, avoiding the interactive scene from changing the content due to the interactive information, so that the lecture will not be disturbed by the improper interactive behavior of the students, thereby improving the quality of the lecture. The first terminal device 100 will only send the interactive information generated by itself, such as interactive information 3, to the cloud server 300 during this period.

[0105] For example Figure 3bAs shown, when the interactive scene is an online game scene with multiple participants, since each participant needs to control the game objects in the game scene in real time and frequently during the game, the interactive permission information during this period is that there is no prohibition on interactive behavior. When the first terminal device 100 receives interactive information such as interactive information 1 and interactive information 2 from the second data connection, it will forward it through the first data connection as soon as possible, so that the cloud server 300 can respond to all interactive information in a timely manner and update the interactive video stream.

[0106] Next, it is described how the first terminal device and the second terminal device establish the second data connection.

[0107] In a possible implementation, after S201: establishing a first data connection with a cloud server by running an interactive application, the method further includes:

[0108] The connection address information of the first terminal device is sent in a network environment corresponding to the second terminal device, wherein the first terminal device and the second terminal device are connected in the network environment, and the connection address information is used to characterize the first terminal device as a main device of the interaction scenario.

[0109] The network environment is a network environment that can be used to carry the data transmission requirements of interactive applications. At least a first terminal device and a second terminal device are connected to the network environment. The environment type of the network environment is not limited in this application. For example, it can be a local area network, a public network, etc., and its environment type may be related to the operating conditions of the interactive application.

[0110] After the first terminal device establishes a first data connection with the cloud server based on the generated interaction scenario, the first terminal device can be connected to the slave device (second terminal device) as the master device of the interaction scenario.

[0111] In order for the second terminal device to find the first terminal device to join the interaction scenario, the first terminal device can send its own connection address information in the network environment, where the connection address information identifies the network address of the first terminal device in the network environment for establishing a second data connection.

[0112] In addition, the connection address information can also identify the first terminal device as the master device of the interactive scene, so that when the second terminal device joins, it can clearly know that it will be a slave device in the interactive scene, that is, all data related to the interactive scene (interactive video stream and interactive information) need to be forwarded through the first terminal device.

[0113] like Figure 4a As shown, the first terminal device 100 can be recorded as a master device, and the second terminal device 200 can be recorded as a slave device.

[0114] After the master device establishes the first data connection with the cloud server 300, it starts to control the cloud server 300 and receive the screen (interactive video stream). The master device publishes its connection address information as a forwarding server to the local area network or the public network, waiting for a slave device to initiate a connection to itself through this connection address information.

[0115] The slave device queries the connection address information published by the master device from the local area network or the public network, and prepares to connect to the master device. After the slave device uses the connection address information to establish a second data connection with the master device, the master device continuously forwards the received interactive video stream to the slave device through the second data connection, and the slave device forwards the interactive information generated based on the interactive behavior to the master device. The interactive information corresponding to the interactive behavior of the master and slave devices will be sent to the cloud server 300 to implement the control response.

[0116] like Figure 4b As shown, the first terminal device 100 can be recorded as a master device, and the second terminal device 200 can be recorded as a slave device.

[0117] When the network environment is a local area network, the master device only needs to send the connection address information to the local area network in the form of local area network broadcast. The connection address information includes the port and Internet Protocol (IP) required to establish the second data connection. If the slave device that establishes intercommunication with the master device is not in the same local area network, it is necessary to use the server of the public network to forward the connection address information and punch a hole in the User Datagram Protocol (UDP) to establish the second data connection.

[0118] The master device uploads the connection address information to the public network, and the slave device receives the connection address information from the server of the public network. The slave device establishes a second data connection with the master device through the connection address information, and starts to send and receive interactive video streams and interactive information.

[0119] Next, it is explained how the first terminal device sends or forwards the interaction information to the cloud server based on itself, so that the cloud server can achieve a correct interaction behavior response.

[0120] In a possible implementation, after establishing the first data connection with the cloud server by running the interactive application, the method further includes:

[0121] A plurality of virtual input component pairs are configured between the first terminal device and the cloud server, the virtual input component pairs are used to forward the interaction information, the virtual input component pairs include a first input component and a second input component, the first input component is configured on the first terminal device, and the second input component is configured on the cloud server.

[0122] The first input component and the second input component in the virtual input component pair are virtualizations of the actual input components, which can be realized through software configuration. Through virtualization, a computer device (such as a first terminal device and a cloud server) can be made to believe that the first terminal device is actually connected to actual input components of various component types. Therefore, the first terminal device and the cloud server will also use the input logic and processing method of the actual input component to process the interactive information received and obtained by the virtual input component pair.

[0123] For example, when the virtual input component pair a (including the first input component a and the second input component a) is a virtualization of a hardware mouse, when the first terminal device obtains the interaction information associated with the virtual input component pair a, when the interaction information is processed by the first input component a, the first terminal device will think that it is the input of the actual input device virtualized by the virtual input component pair a. The first terminal device will use the processing logic corresponding to the hardware mouse to send the interaction information to the cloud server through the first data connection based on the first input component a. When the cloud server determines that the interaction information is associated with the second input component a, it will use the processing logic of the hardware mouse to process the interaction information through the second input component a, so that the cloud server can correctly understand the interaction information.

[0124] Due to the interactive scenario involving multiple people, the first terminal device acts as a transit device to establish a first data connection with the cloud server alone, and the interactive information comes not only from itself, but also from the second terminal device. However, the interactive information obtained by the cloud server is all sent by the first terminal device through the first data connection. Therefore, the first terminal device needs to instruct the cloud server in a clear manner how to respond to different interactive information.

[0125] In this implementation, accurate identification of the source of the interactive information is achieved through pre-paired virtual input components, and a virtual component pair includes a first input component and a second input component, which are respectively configured in the first terminal device and the cloud server.

[0126] After configuring the virtual component pair, the first terminal device can establish an association between the terminal device (e.g., the first terminal device and the second terminal device) and the virtual component pair. Thus, a virtual input component pair can be used to forward the interactive information of the terminal device associated with it, thereby achieving accurate distinction of the interactive information and improving the response accuracy of the cloud server to the interactive information.

[0127] Next, the use of the virtual component pair of the present application in forwarding interactive information is explained by taking the second terminal device and the target component pair associated therewith as an example.

[0128] In a possible implementation, in S203: forwarding the interaction information obtained from the second data connection to the cloud server through the first data connection includes:

[0129] S11: Acquire the component type of the actual input component of the second terminal device;

[0130] S12: determining a target component pair from the plurality of virtual input component pairs according to the component type, and establishing an association relationship between the second terminal device and the target component pair;

[0131] S13: when acquiring the interaction information from the second data connection, determining the target component pair according to the device identification of the second terminal device and the association relationship;

[0132] S14: forwarding the interaction information and the pair identifier of the target component pair from the first data connection to the cloud server through the first input component of the target component pair, wherein the pair identifier is used to instruct the cloud server to respond to the interaction information based on the second input component of the target component pair.

[0133] When the terminal device is actually used by the user to implement interactive behaviors for interactive scenarios, the interactive behaviors are responded to through the actual input components of the terminal device itself. The component type of the actual input component can be, for example, virtual controls on a touch screen, hardware keyboard and mouse, handles, etc. Actual input components of different component types have different input logic and processing methods.

[0134] The virtual component pair simulates the actual input component in a virtual way to "cheat" the computer device, such as the cloud server or the first terminal device, so that the computer device believes that the first terminal device is indeed equipped with actual input components of various component types. In other words, the virtual component pair also needs to simulate the input logic and processing methods of actual input components of different component types.

[0135] Therefore, the first terminal device needs to determine an adapted target component pair from the virtual input component pair based on the component type of the actual input component of the second terminal device to establish an association relationship with the second terminal device. The determined target component pair is the virtual input component pair of the actual input component of the virtual second terminal device.

[0136] The association relationship may be determined when the second data connection is established, or may be determined when the interaction information is acquired from the second data connection, and this application does not limit this.

[0137] After the association relationship is established, the association relationship generally does not change while the second terminal device remains in the interaction scenario, unless the second terminal device is disconnected from the network, etc. The first terminal device will re-determine the association relationship after the second terminal device reconnects, or reuse the original association relationship.

[0138] It should be noted that, in addition to determining the target component pairs associated with each second terminal device, the first terminal device may also determine the target component pairs associated with its own actual input components, so as to unify the format of sending or forwarding interactive information.

[0139] Since all interactive information that needs to be sent to the cloud server is transmitted through the first data connection, in order to allow the cloud server to clearly identify the source of the interactive information obtained from the first data connection, the first terminal device will also carry the pair identifier of the virtual component pair used when sending or forwarding the interactive information. Since the two input components in the virtual component pair share the pair identifier, when the cloud server obtains an interactive information, it can accurately find the second input component of the virtual component pair through the pair identifier.

[0140] like Figure 5 As shown, between the first terminal device and the cloud server, the first terminal device is configured with n virtual input component pairs, each virtual input component pair includes two input components, for example, virtual input component pair 1 includes a first input component 1 and a second input component 1, virtual input component pair 2 includes a first input component 2 and a second input component 2, and virtual input component pair n includes a first input component n and a second input component n.

[0141] The first input component in each virtual input component pair is configured in the first terminal device, and the second input component in each virtual input component pair is configured in the cloud server.

[0142] After the first terminal device establishes the first data connection with the cloud server, n virtual input component pairs can be configured to provide the second terminal device that joins the interaction scene with the purpose of forwarding interaction information.

[0143] On the cloud server side, a peripheral driver can be installed, which can be used to configure a virtual input component, such as the second input component in the virtual input component pair. Therefore, when the cloud server obtains the interaction information through the first data connection, it can send it to the operating system of the cloud server through the peripheral driver based on the corresponding second input component, thereby realizing the input of multiple terminal devices in a software manner.

[0144] In the embodiment of the present application, the flow of an interactive message may be as follows:

[0145] When the main device obtains the interaction information through the second data connection, or generates the interaction information in response to its own interaction behavior, the interaction information is sent to the client of the locally installed interaction application through the operating system of the main device.

[0146] The client of the master device determines the virtual input component pair corresponding to the interaction information based on the association relationship, and then sends the interaction information to the cloud server through the first data connection based on the first input component of the virtual input component pair (e.g., the first input component 2 of the virtual input component pair 2). When sending, the interaction information and the pair identifier of the virtual input component pair (e.g., the pair identifier of the virtual input component pair 2) can be encapsulated according to the input protocol (which can be a private protocol).

[0147] After the cloud server obtains the encapsulated package of the interaction information, it parses it through the same input protocol to obtain the interaction information and pair identification. Since the pair identification serves as a unique identifier of the first input component and the second input component in the corresponding virtual input component pair, the cloud server can determine the corresponding second input component (for example, the second input component 2 of virtual input component pair 2) based on the pair identification, and send the interaction information to the second input component.

[0148] The second input component processes the interaction information through the peripheral driver and sends it to the operating system of the cloud server. Since the interaction information sent by the peripheral driver to the operating system conforms to the input peripheral specification of the operating system, it can be regarded as ordinary hardware device input by the operating system and respond correctly, thereby realizing the ability to complete multi-user input through the first terminal device.

[0149] In a possible implementation, the multiple virtual input component pairs of the present application may have multiple configurations.

[0150] The first method: Pre-configuration

[0151] In this implementation, after establishing a first data connection with the cloud server, the first terminal device pre-configures a sufficient number of virtual input component pairs according to the cloud server's adaptation range for input peripherals and the requirements of the interaction scene. Through pre-configuration, during the duration of the subsequent interaction scene, there is no need to configure the virtual input component pairs for the second terminal device that joins later, thereby improving the processing efficiency of the subsequent interaction scene.

[0152] Specifically, the aforementioned configuration of multiple virtual input component pairs between the first terminal device and the cloud server includes:

[0153] S21: Determine a component type set of actual input components adapted by the cloud server;

[0154] S22: Before obtaining the interaction information from the second data connection, configuring virtual input component pairs corresponding to the component types in the component type set in the first terminal device and the cloud server.

[0155] That is to say, after the first terminal device determines the component type set of actual input components that the cloud server can adapt, such as supporting keyboard and mouse but not supporting handles, it can configure matching virtual input component pairs on the first terminal device and the cloud server based on the component types involved in the component type set, such as 10 virtual input component pairs. When a second terminal device subsequently joins the interaction scene, it can directly select the adapted virtual input component pair based on the component type of the actual input component of the second terminal device to establish an association relationship.

[0156] The second method: temporary configuration

[0157] In this implementation, only when the second terminal device is actually added to the interactive scene, a matching virtual input component pair will be configured based on the component type of the actual input component of the second terminal device. In this way, the utilization efficiency of the virtual input component pairs can be improved, and the waste of maintenance resources caused by some virtual input component pairs being idle for a long time can be avoided, thereby improving resource utilization efficiency.

[0158] Specifically, for the second terminal device, the aforementioned configuration of multiple virtual input component pairs between the first terminal device and the cloud server includes:

[0159] Acquire the component type of the actual input component of the second terminal device;

[0160] According to the component type, a virtual input component pair corresponding to the component type is configured between the first terminal device and the cloud server.

[0161] Next, the data processing solution of this application will be further explained through the cloud server side.

[0162] Figure 6 A method flow chart of another data processing method provided in an embodiment of the present application, wherein the method is executed by a cloud server, and the method includes:

[0163] S601: In response to a first terminal device running an interactive application, a first data connection with the first terminal device is established, and an interactive scenario of the interactive application is generated.

[0164] The interaction scene may include multiple interaction objects.

[0165] S602: Acquire interaction information sent by the first terminal device through the first data connection.

[0166] S603: Determine a target object corresponding to the interaction information from the multiple interaction objects.

[0167] S604: Control the target object to perform corresponding interactive behavior in the interactive scene according to the interactive information, and send an interactive video stream corresponding to the interactive scene to the first terminal device through the first data connection.

[0168] Because the cloud server only connects to the first terminal device when providing processing services for interactive scenarios involving multiple people, and does not additionally perceive other terminal devices that join the interactive scene, such as the second terminal device, the cloud server needs to clearly understand which interactive object in the interactive scene the interactive information sent from the first data connection is related to, so that it can make a correct response.

[0169] After the cloud server determines the target object corresponding to the current interaction information from multiple interaction objects, it can control the target object through the interaction information to perform the interaction behavior indicated by the interaction information in the interaction scene.

[0170] It can be seen that the cloud server only needs to send the interactive video stream to the first terminal device, and does not need to send the interactive video stream to other terminal devices, which greatly reduces the bandwidth pressure of the cloud server.

[0171] In a possible implementation, the cloud server is configured with multiple second input components, different second input components belong to different virtual input component pairs, the virtual input component pair is used to forward the interaction information, the virtual input component pair includes a first input component and the second input component, and the first input component is configured on the first terminal device.

[0172] In a possible implementation manner, S602: acquiring, through the first data connection, interaction information sent by the first terminal device, includes:

[0173] Acquire the interaction information and the pair identifier sent by the first terminal device through the first data connection;

[0174] S603: Determining a target object corresponding to the interaction information from the multiple interaction objects, including:

[0175] According to the pair identification association information pre-acquired from the first terminal device, a target component pair associated with the pair identification is determined from the multiple virtual input component pairs, the pair identification association information is used to characterize the pair identifications corresponding to the multiple virtual input component pairs respectively, and the target component pair has a corresponding relationship with the target object.

[0176] The cloud server can determine the second input component in the corresponding target component pair according to the pair identifier sent together with the interaction information, and then control the target object to perform corresponding interaction behavior in the interaction scene according to the association relationship between the second input component and the interaction object.

[0177] In the embodiments related to the cloud server, unexplained features can refer to the description in the previous embodiments and will not be repeated here.

[0178] Figure 7a One of the terminal device transfer schematic diagrams based on a virtual input component pair provided in an embodiment of the present application. In this embodiment, the first terminal device establishes a first data connection with the cloud server as a master device, and the second terminal device establishes a second data connection with the first terminal device as a slave device.

[0179] N virtual input component pairs are configured between the first terminal device and the cloud server. Exemplarily, virtual input component pair 1 (including first input component 1 and second input component 1) is associated with the first terminal device, and virtual input component pair 2 (including first input component 2 and second input component 2) is associated with the second terminal device.

[0180] In this embodiment, the interaction permission information indicates that there is no prohibition on interaction behavior. Assuming that the first terminal device generates interaction information b while receiving interaction information a from the second terminal device, the first terminal device will send interaction information b through the first input component 1 and interaction information a through the first input component 2. When the cloud server obtains interaction information a and interaction information b, it sends interaction information b to the second input component 1 for processing and interaction information a to the second input component 2 for processing through the carried pair identifier. After processing, interaction information a and interaction information b will be sent to the operating system of the cloud server for control response.

[0181] Figure 7b The second schematic diagram of a terminal device transfer based on a virtual input component pair provided in an embodiment of the present application. In this embodiment, the first terminal device as a master device establishes a first data connection with a cloud server, and the second terminal device as a slave device establishes a second data connection with the first terminal device.

[0182] N virtual input component pairs are configured between the first terminal device and the cloud server. Exemplarily, virtual input component pair 1 (including first input component 1 and second input component 1) is associated with the first terminal device, and virtual input component pair 2 (including first input component 2 and second input component 2) is associated with the second terminal device.

[0183] In this embodiment, the interaction permission information prohibits the interaction behavior of the first terminal device. Assume that the first terminal device generates interaction information b while receiving interaction information a from the second terminal device. Based on the interaction permission information, the first terminal device will not process the interaction information b, or avoid sending the interaction information b to the cloud server by temporarily deactivating the first input component 1.

[0184] The first terminal device sends the interaction information a through the first input component 2. When the cloud server obtains the interaction information a, it sends the interaction information a to the second input component 2 for processing through the carried pair identifier. After the processing, the interaction information a will be sent to the operating system of the cloud server for control response.

[0185] Figure 8 A device structure diagram of a first terminal device provided in an embodiment of the present application, wherein the first terminal device 800 includes an establishing unit 801 and a forwarding unit 802:

[0186] The establishing unit 801 is used to establish a first data connection with the cloud server by running an interactive application;

[0187] The establishing unit 801 is further configured to establish a second data connection with the second terminal device in response to a connection request of the second terminal device, and the second terminal device and the first terminal device jointly participate in the interactive scenario of the interactive application;

[0188] The forwarding unit 802 is used to forward the interactive video stream obtained from the first data connection to the second terminal device through the second data connection, and forward the interactive information obtained from the second data connection to the cloud server through the first data connection, the interactive video stream is used to display the interactive scene, and the interactive information is used to characterize the interactive behavior based on the second terminal device on the interactive scene.

[0189] In a possible implementation manner, the forwarding unit is further configured to:

[0190] In response to acquiring the interaction information from the second data connection, determining interaction permission information for the interaction scenario;

[0191] In response to determining that the second terminal device has the interaction authority identified by the interaction authority information, the interaction information is forwarded to the cloud server through the first data connection.

[0192] In a possible implementation manner, the first terminal device further includes a control unit, and the control unit is configured to:

[0193] In response to determining that the second terminal device does not have the interaction authority identified by the interaction authority information, forwarding the interaction information to the cloud server is prohibited.

[0194] In a possible implementation manner, the first terminal device further includes a sending unit, and the sending unit is configured to:

[0195] The connection address information of the first terminal device is sent in a network environment corresponding to the second terminal device, wherein the first terminal device and the second terminal device are connected in the network environment, and the connection address information is used to characterize the first terminal device as a main device of the interaction scenario.

[0196] In a possible implementation manner, the first terminal device further includes a configuration unit, and the configuration unit is configured to:

[0197] A plurality of virtual input component pairs are configured between the first terminal device and the cloud server, the virtual input component pairs are used to forward the interaction information, the virtual input component pairs include a first input component and a second input component, the first input component is configured on the first terminal device, and the second input component is configured on the cloud server.

[0198] In a possible implementation manner, the forwarding unit is further configured to:

[0199] Acquire the component type of the actual input component of the second terminal device;

[0200] Determine a target component pair from among the plurality of virtual input component pairs according to the component type, and establish an association relationship between the second terminal device and the target component pair;

[0201] When acquiring the interaction information from the second data connection, determining the target component pair according to the device identification of the second terminal device and the association relationship;

[0202] The interaction information and the pair identifier of the target component pair are forwarded from the first data connection to the cloud server through the first input component of the target component pair, wherein the pair identifier is used to instruct the cloud server to respond to the interaction information based on the second input component of the target component pair.

[0203] In a possible implementation manner, the configuration unit is further configured to:

[0204] Determine a component type set of actual input components adapted by the cloud server;

[0205] Before the interaction information is obtained from the second data connection, virtual input component pairs corresponding to the component types in the component type set are configured in the first terminal device and the cloud server.

[0206] In a possible implementation manner, for the second terminal device, the configuration unit is further configured to:

[0207] Acquire the component type of the actual input component of the second terminal device;

[0208] According to the component type, a virtual input component pair corresponding to the component type is configured between the first terminal device and the cloud server.

[0209] For the relevant description of the features in this embodiment, please refer to the aforementioned embodiments and will not be repeated here.

[0210] It can be seen that for interactive applications based on cloud servers, when multiple terminal devices are required to jointly participate in the interactive scene of the interactive application, a first data connection with the cloud server can be established based on the first terminal device among the multiple terminal devices by running the interactive application. When the second terminal device among the multiple terminal devices needs to join the interactive scene, the second terminal device directly establishes a second data connection with the first terminal device, and the first terminal device acts as the data transfer role between the cloud server and other terminal devices in the interactive scene. Based on the established first data connection and second data connection, the first terminal device forwards the interactive video stream of the interactive scene provided by the cloud server to other terminal devices participating in the interactive scene, and forwards the interactive information generated by other terminal devices for the interactive scene to the cloud server, so that the cloud server only connects to one terminal device when providing processing services for the interactive scene, that is, it only needs to send the interactive video stream to the first terminal device, and does not need to send the interactive video stream to other terminal devices, which greatly reduces the bandwidth pressure of the cloud server. Moreover, since the interactive video stream can reach the second terminal device through only one layer of real-time forwarding of the first terminal device, the interactive scene displayed based on the interactive video stream on the second terminal device can also achieve good synchronization and real-time performance with the first terminal device, ensuring the user's interactive experience.

[0211] Fig. 9 A device structure diagram of a cloud server provided in an embodiment of the present application, wherein the cloud server 900 includes an establishing unit 901, an acquiring unit 902, a determining unit 903, and a sending unit 904:

[0212] The establishing unit 901 is configured to establish a first data connection with the first terminal device in response to the first terminal device running the interactive application, and generate an interactive scene of the interactive application, wherein the interactive scene includes a plurality of interactive objects;

[0213] The acquiring unit 902 is configured to acquire the interaction information sent by the first terminal device through the first data connection;

[0214] The determining unit 903 is configured to determine a target object corresponding to the interaction information from among the multiple interaction objects;

[0215] The sending unit 904 is used to control the target object to perform corresponding interactive behavior in the interactive scene according to the interactive information, and send the interactive video stream corresponding to the interactive scene to the first terminal device through the first data connection.

[0216] In a possible implementation, the cloud server is configured with multiple second input components, different second input components belong to different virtual input component pairs, the virtual input component pair is used to forward the interaction information, the virtual input component pair includes a first input component and the second input component, and the first input component is configured on the first terminal device.

[0217] In a possible implementation manner, the acquiring unit is further configured to:

[0218] Acquire the interaction information and the pair identifier sent by the first terminal device through the first data connection;

[0219] The determining, from the multiple interaction objects, a target object corresponding to the interaction information includes:

[0220] According to the pair identification association information pre-acquired from the first terminal device, a target component pair associated with the pair identification is determined from the multiple virtual input component pairs, the pair identification association information is used to characterize the pair identifications corresponding to the multiple virtual input component pairs respectively, and the target component pair has a corresponding relationship with the target object.

[0221] For the relevant description of the features in this embodiment, please refer to the aforementioned embodiments and will not be repeated here.

[0222] It can be seen that the cloud server only needs to send the interactive video stream to the first terminal device, and does not need to send the interactive video stream to other terminal devices, which greatly reduces the bandwidth pressure of the cloud server.

[0223] Fig.10 This is a structural diagram of a data processing system provided in an embodiment of the present application. The data processing system 1000 includes a first terminal device 1001 and a cloud server 1002:

[0224] The first terminal device 1001 is used to execute the method executed by the first terminal device in the above embodiment;

[0225] The cloud server 1002 is used to execute the method performed by the cloud server in the aforementioned embodiment.

[0226] The embodiment of the present application further provides a computer device, which is the computer device described above, and the first terminal device or cloud server described above can serve as the computer device. The computer device is described below in conjunction with the accompanying drawings.

[0227] If the computer device is a terminal device such as the first terminal device mentioned above, please refer to Fig.11 As shown, the embodiment of the present application provides a terminal device, taking the terminal device as a mobile phone as an example:

[0228] Fig.11 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Fig.11 The mobile phone includes: a radio frequency (RF) circuit 1410, a memory 1420, an input unit 1430, a display unit 1440, a sensor 1450, an audio circuit 1460, a wireless fidelity (WiFi) module 1470, a processor 1480, and a power supply 1490. Those skilled in the art can understand that Fig.11 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0229] Combine the following Fig.11 A detailed introduction to the various components of the mobile phone:

[0230] The RF circuit 1410 may be used for receiving and sending signals during information transmission or communication or during a call. In particular, after receiving downlink information from a base station, the information is sent to the processor 1480 for processing. In addition, the uplink data is sent to the base station.

[0231] The memory 1420 can be used to store software programs and modules. The processor 1480 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1420. The memory 1420 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 1420 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0232] The input unit 1430 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the mobile phone. Specifically, the input unit 1430 may include a touch panel 1431 and other input devices 1432 .

[0233] The display unit 1440 may be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1440 may include a display panel 1441.

[0234] The mobile phone may also include at least one sensor 1450, such as a light sensor, a motion sensor, and other sensors.

[0235] The audio circuit 1460 , the speaker 1461 , and the microphone 1462 can provide an audio interface between the user and the mobile phone.

[0236] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 1470. It provides users with wireless broadband Internet access.

[0237] Processor 1480 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in memory 1420 and calling data stored in memory 1420.

[0238] The mobile phone also includes a power source 1490 (such as a battery) for supplying power to various components.

[0239] In this embodiment, the processor 1480 included in the terminal device also has the following functions:

[0240] Establishing a first data connection with the cloud server by running an interactive application;

[0241] In response to a connection request from a second terminal device, a second data connection is established with the second terminal device, and the second terminal device and the first terminal device jointly participate in an interactive scenario of the interactive application;

[0242] The interactive video stream obtained from the first data connection is forwarded to the second terminal device via the second data connection, and the interactive information obtained from the second data connection is forwarded to the cloud server via the first data connection, wherein the interactive video stream is used to display the interactive scene, and the interactive information is used to characterize the interactive behavior performed on the interactive scene by the second terminal device.

[0243] Other steps performed by the terminal device in the above embodiment may also be based on Fig.11The terminal device structure shown.

[0244] If the computer device is a server such as the aforementioned cloud server, the present application embodiment also provides a server, see Fig.12 As shown, Fig.12 The structural diagram of the server 1500 provided in the embodiment of the present application, the server 1500 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1522 (for example, one or more processors) and memory 1532, one or more storage media 1530 (for example, one or more mass storage devices) storing application programs 1542 or data 1544. Among them, the memory 1532 and the storage medium 1530 can be short-term storage or persistent storage. The program stored in the storage medium 1530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 1522 can be configured to communicate with the storage medium 1530 to execute a series of instruction operations in the storage medium 1530 on the server 1500.

[0245] The server 1500 may also include one or more power supplies 1526, one or more wired or wireless network interfaces 1550, one or more input and output interfaces 1558, and / or one or more operating systems 1541, such as Windows Server 2003 or Windows Server 2008. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0246] In this embodiment, the processor 1522 included in the server also has the following functions:

[0247] In response to the first terminal device running the interactive application, establishing a first data connection with the first terminal device, and generating an interactive scene of the interactive application, wherein the interactive scene includes a plurality of interactive objects;

[0248] Acquire interaction information sent by the first terminal device through the first data connection;

[0249] Determine a target object corresponding to the interaction information from the multiple interaction objects;

[0250] The target object is controlled to perform corresponding interactive behavior in the interactive scene according to the interactive information, and an interactive video stream corresponding to the interactive scene is sent to the first terminal device through the first data connection.

[0251] Other steps performed by the server in the above embodiment may also be based on Fig.12 The server structure shown.

[0252] In addition, an embodiment of the present application further provides a storage medium, wherein the storage medium is used to store a computer program, and the computer program is used to execute the method provided in the above embodiment.

[0253] The embodiments of the present application also provide a computer program product including a computer program, which, when executed on a computer device, enables the computer device to execute the method provided in the above embodiments.

[0254] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the above program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the above storage medium can be at least one of the following media: read-only memory (English: Read-only Memory, abbreviated: ROM), RAM, magnetic disk or optical disk, etc. Various media that can store computer programs.

[0255] It should be noted that each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0256] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a technician familiar with the technical field within the technical scope disclosed in the present application should be included in the protection scope of the present application. Moreover, based on the implementation methods provided in the above aspects, the present application can also be further combined to provide more implementation methods. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: The method is performed by a first terminal device, and the method includes: Establishing a first data connection with the cloud server by running an interactive application; In response to a connection request from a second terminal device, a second data connection is established with the second terminal device, and the second terminal device and the first terminal device jointly participate in an interactive scenario of the interactive application; The interactive video stream obtained from the first data connection is forwarded to the second terminal device via the second data connection, and the interactive information obtained from the second data connection is forwarded to the cloud server via the first data connection, wherein the interactive video stream is used to display the interactive scene, and the interactive information is used to characterize the interactive behavior performed on the interactive scene by the second terminal device.

2. The method according to claim 1, characterized in that The forwarding the interaction information obtained from the second data connection to the cloud server through the first data connection includes: In response to acquiring the interaction information from the second data connection, determining interaction permission information for the interaction scenario; In response to determining that the second terminal device has the interaction authority identified by the interaction authority information, the interaction information is forwarded to the cloud server through the first data connection.

3. The method according to claim 2, characterized in that The method further comprises: In response to determining that the second terminal device does not have the interaction authority identified by the interaction authority information, forwarding the interaction information to the cloud server is prohibited.

4. The method according to claim 1, characterized in that After establishing the first data connection with the cloud server by running the interactive application, the method further includes: The connection address information of the first terminal device is sent in a network environment corresponding to the second terminal device, wherein the first terminal device and the second terminal device are connected in the network environment, and the connection address information is used to characterize the first terminal device as a main device of the interaction scenario.

5. The method according to claim 1, characterized in that After establishing the first data connection with the cloud server by running the interactive application, the method further includes: A plurality of virtual input component pairs are configured between the first terminal device and the cloud server, the virtual input component pairs are used to forward the interaction information, the virtual input component pairs include a first input component and a second input component, the first input component is configured on the first terminal device, and the second input component is configured on the cloud server.

6. The method according to claim 5, characterized in that The forwarding the interaction information obtained from the second data connection to the cloud server through the first data connection includes: Acquire the component type of the actual input component of the second terminal device; Determine a target component pair from among the plurality of virtual input component pairs according to the component type, and establish an association relationship between the second terminal device and the target component pair; When acquiring the interaction information from the second data connection, determining the target component pair according to the device identification of the second terminal device and the association relationship; The interaction information and the pair identifier of the target component pair are forwarded from the first data connection to the cloud server through the first input component of the target component pair, wherein the pair identifier is used to instruct the cloud server to respond to the interaction information based on the second input component of the target component pair.

7. The method according to claim 5, characterized in that The configuring of a plurality of virtual input component pairs between the first terminal device and the cloud server comprises: Determine a component type set of actual input components adapted by the cloud server; Before the interaction information is obtained from the second data connection, virtual input component pairs corresponding to the component types in the component type set are configured in the first terminal device and the cloud server.

8. The method according to claim 5, characterized in that For the second terminal device, configuring a plurality of virtual input component pairs between the first terminal device and the cloud server includes: Acquire the component type of the actual input component of the second terminal device; According to the component type, a virtual input component pair corresponding to the component type is configured between the first terminal device and the cloud server.

9. A data processing method, characterized in that: The method is performed by a cloud server, and includes: In response to the first terminal device running the interactive application, establishing a first data connection with the first terminal device, and generating an interactive scene of the interactive application, wherein the interactive scene includes a plurality of interactive objects; Acquire interaction information sent by the first terminal device through the first data connection; Determine a target object corresponding to the interaction information from the multiple interaction objects; The target object is controlled to perform corresponding interactive behavior in the interactive scene according to the interactive information, and an interactive video stream corresponding to the interactive scene is sent to the first terminal device through the first data connection.

10. The method according to claim 9, characterized in that The cloud server is configured with multiple second input components, different second input components belong to different virtual input component pairs, the virtual input component pair is used to forward the interaction information, the virtual input component pair includes a first input component and the second input component, and the first input component is configured on the first terminal device.

11. The method according to claim 10, characterized in that The acquiring, through the first data connection, the interaction information sent by the first terminal device includes: Acquire the interaction information and the pair identifier sent by the first terminal device through the first data connection; The determining, from the multiple interaction objects, a target object corresponding to the interaction information includes: According to the pair identification association information pre-acquired from the first terminal device, a target component pair associated with the pair identification is determined from the multiple virtual input component pairs, the pair identification association information is used to characterize the pair identifications corresponding to the multiple virtual input component pairs respectively, and the target component pair has a corresponding relationship with the target object.

12. A first terminal device, characterized in that: The first terminal device includes an establishing unit and a forwarding unit: The establishing unit is used to establish a first data connection with the cloud server by running an interactive application; The establishing unit is further configured to establish a second data connection with the second terminal device in response to a connection request of the second terminal device, wherein the second terminal device and the first terminal device jointly participate in the interactive scenario of the interactive application; The forwarding unit is used to forward the interactive video stream obtained from the first data connection to the second terminal device through the second data connection, and forward the interactive information obtained from the second data connection to the cloud server through the first data connection, the interactive video stream is used to display the interactive scene, and the interactive information is used to characterize the interactive behavior based on the second terminal device on the interactive scene.

13. A cloud server, characterized in that: The cloud server includes an establishing unit, an acquiring unit, a determining unit and a sending unit: The establishing unit is configured to establish a first data connection with the first terminal device in response to the first terminal device running the interactive application, and generate an interactive scene of the interactive application, wherein the interactive scene includes a plurality of interactive objects; The acquiring unit is configured to acquire the interaction information sent by the first terminal device through the first data connection; The determining unit is used to determine a target object corresponding to the interaction information from the multiple interaction objects; The sending unit is used to control the target object to perform corresponding interactive behavior in the interactive scene according to the interactive information, and send the interactive video stream corresponding to the interactive scene to the first terminal device through the first data connection.

14. A data processing system, characterized in that: The system includes a first terminal device and a cloud server: The first terminal device is used to execute the method according to any one of claims 1 to 8; The cloud server is used to execute the method according to any one of claims 9 to 11.

15. A computer device, characterized in that: The computer device comprises a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is used to execute the method described in any one of claims 1 to 8 according to the computer program, or to execute the method described in any one of claims 9 to 11.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a computer device, the computer program implements the method described in any one of claims 1 to 8, or the method described in any one of claims 9 to 11.

17. A computer program product comprising a computer program, which, when executed on a computer device, enables the computer device to execute the method according to any one of claims 1 to 8, or the method according to any one of claims 9 to 11.

Images