An information processing method, device and readable storage medium

Abstract

The application provides an information processing method, device and readable storage medium, wherein the information processing method comprises: obtaining at least one first information according to a sharing request; and sending the sharing request and the first information to a server; wherein the sharing request is a sharing request for at least a first part of screen pictures of the first terminal. The embodiment of the application can support the conversion from the case of multiple people operating one terminal to play a game to the case of multiple people operating multiple terminals to play a game, thereby meeting the diversified needs of users, facilitating the use of users, fully utilizing the terminals around the users, and well solving the problem that the prior art cannot meet the diversified needs of users in the information processing scheme for multiple cloud games.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to an information processing method, apparatus, and readable storage medium. Background Technology

[0002] In existing multiplayer cloud games, multiple users can only operate one game terminal at a time, which cannot meet the diverse needs of users, is inconvenient for users, and cannot make full use of the terminals available to them.

[0003] As shown above, existing information processing solutions for multiplayer cloud games have shortcomings such as failing to meet the diverse needs of users. Summary of the Invention

[0004] The purpose of this invention is to provide an information processing method, device, and readable storage medium to solve the problem that existing information processing solutions for multiplayer cloud games cannot meet the diverse needs of users.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide an information processing method applied to a first terminal, comprising:

[0006] Based on the sharing request, obtain at least one piece of initial information;

[0007] Send the sharing request and the first information to the server;

[0008] The sharing request is a sharing request for at least a first portion of the screen of the first terminal.

[0009] Optionally, after sending the sharing request and the first information to the server, the method further includes:

[0010] Stop sending the operation instructions corresponding to the first information received to the cloud device.

[0011] Optional, also includes:

[0012] Based on the merge request, obtain at least one piece of second information;

[0013] Send the merge request and the second information to the server;

[0014] The merging request is a merging request for at least a second portion of the screen display of the first terminal.

[0015] Optionally, after sending the merge request and the second information to the server, the method further includes:

[0016] Send the operation command corresponding to the received second information to the cloud device.

[0017] This invention also provides an information processing method applied to a server, comprising:

[0018] Receive a sharing request and at least one first piece of information sent by the first terminal;

[0019] Based on the first information, send third information regarding the sharing request to the cloud device;

[0020] The sharing request is a sharing request for at least a first portion of the screen of the first terminal.

[0021] The third information includes: the terminal identification information of the second terminal;

[0022] The second terminal is the terminal associated with the first information.

[0023] Optionally, sending third information regarding the sharing request to the cloud device based on the first information includes:

[0024] Based on the first information, the corresponding second terminal is determined;

[0025] Send the sharing request to the second terminal;

[0026] The system receives first response information from the second terminal, the first response information including: a first response information in which the second terminal agrees to the sharing request;

[0027] The first response information and the terminal identification information of the second terminal are sent to the cloud device as the third information.

[0028] Optionally, the first information includes: second identification information;

[0029] The step of determining the corresponding second terminal based on the first information includes:

[0030] The corresponding second terminal is determined based on the second identification information.

[0031] Optionally, the first response information may further include at least one of the resolution and bandwidth corresponding to the second terminal.

[0032] Optional, also includes:

[0033] Receive a merging request and at least one second piece of information sent by the first terminal;

[0034] Based on the second information, send the fourth information regarding the merging request to the cloud device;

[0035] The merging request is a merging request for at least a second portion of the screen image of the first terminal.

[0036] The fourth piece of information includes: the terminal identification information of the third terminal;

[0037] The third terminal is the terminal associated with the second information.

[0038] Optionally, sending fourth information regarding the merging request to the cloud device based on the second information includes:

[0039] Based on the second information, the corresponding third terminal is determined;

[0040] The merge request is sent to the third terminal;

[0041] Receive a second response from the third terminal indicating agreement to the merger request;

[0042] The second response information and the terminal identification information of the third terminal are sent to the cloud device as the fourth information.

[0043] Optionally, the second information includes: first identification information;

[0044] The step of determining the corresponding third terminal based on the second information includes:

[0045] Based on the first identification information, the corresponding third terminal is determined.

[0046] This invention also provides an information processing method applied to cloud devices, comprising:

[0047] The server receives third information in response to a sharing request; the sharing request is a sharing request for at least a first portion of the screen of the first terminal; the third information includes: terminal identification information of the second terminal;

[0048] A connection is established with the second terminal based on the terminal identification information of the second terminal;

[0049] Through the connection, data concerning at least the first portion of the screen image is sent to the second terminal.

[0050] Optionally, the third information further includes: first response information; the first response information includes: a first response information indicating that the second terminal agrees to the sharing request;

[0051] The step of establishing a connection with the second terminal based on the terminal identification information of the second terminal includes:

[0052] A connection is established with the second terminal based on the terminal identification information and the first response information.

[0053] Optionally, the first response information may further include at least one of the resolution and bandwidth corresponding to the second terminal;

[0054] Sending data for at least the first portion of the screen display to the second terminal via the connection includes:

[0055] Based on the resolution and bandwidth, determine the maximum data transmission rate corresponding to the second terminal;

[0056] Based on the resolution, bandwidth, and maximum bitrate, data for at least the first portion of the screen image is sent to the second terminal via the connection.

[0057] Optionally, sending data regarding the at least first portion of the screen display to the second terminal via the connection includes:

[0058] Through the connection, the system receives operation instructions sent by the second terminal for at least the first portion of the screen display.

[0059] According to the operation instructions sent by the second terminal, data for the at least first part of the screen image is sent to the second terminal through the connection.

[0060] Optional, also includes:

[0061] The server sends a fourth piece of information in response to the merging request; the merging request is a merging request for at least a second portion of the screen of the first terminal; the fourth piece of information includes: the terminal identification information of the third terminal;

[0062] The connection with the third terminal is disconnected based on the terminal identification information of the third terminal.

[0063] Optionally, the fourth information further includes: a second response information indicating that the third terminal agrees to the merging request;

[0064] Disconnecting from the third terminal based on the terminal identification information of the third terminal includes:

[0065] Based on the terminal identification information and the second response information of the third terminal, the connection with the third terminal is disconnected.

[0066] Optionally, after disconnecting the connection with the third terminal based on the terminal identification information of the third terminal, the method further includes:

[0067] Based on the terminal identification information of the third terminal, the first identification information is determined;

[0068] Based on the first identification information and the first mapping relationship, it is determined that the first control device is connected to the first terminal;

[0069] Wherein, the first mapping relationship is the mapping relationship between the first identification information and the first control device that controls the at least second part of the screen.

[0070] Optionally, after determining that the first control device is connected to the first terminal based on the first identification information and the first mapping relationship, the method further includes:

[0071] Process the operation instructions corresponding to the second information sent by the first terminal;

[0072] The second information includes the first identification information;

[0073] The second information is obtained by the first terminal based on the merging request.

[0074] This invention also provides an information processing device applied to a first terminal, comprising:

[0075] The first acquisition module is used to acquire at least one piece of first information based on the sharing request;

[0076] The first sending module is used to send the sharing request and the first information to the server;

[0077] The sharing request is a sharing request for at least a first portion of the screen of the first terminal.

[0078] Optional, also includes:

[0079] The first processing module is configured to stop sending the operation instructions corresponding to the received first information to the cloud device after sending the sharing request and the first information to the server.

[0080] Optional, also includes:

[0081] The second acquisition module is used to acquire at least one piece of second information based on the merge request;

[0082] The second sending module is used to send the merge request and the second information to the server;

[0083] The merging request is a merging request for at least a second portion of the screen display of the first terminal.

[0084] Optional, also includes:

[0085] The third sending module is used to send the operation instruction corresponding to the received second information to the cloud device after sending the merge request and the second information to the server.

[0086] This invention also provides an information processing apparatus applied to a server, comprising:

[0087] The first receiving module is used to receive a sharing request and at least one first piece of information sent by the first terminal;

[0088] The fourth sending module is used to send third information in response to the sharing request to the cloud device based on the first information;

[0089] The sharing request is a sharing request for at least a first portion of the screen of the first terminal.

[0090] The third information includes: the terminal identification information of the second terminal;

[0091] The second terminal is the terminal associated with the first information.

[0092] Optionally, sending third information regarding the sharing request to the cloud device based on the first information includes:

[0093] Based on the first information, the corresponding second terminal is determined;

[0094] Send the sharing request to the second terminal;

[0095] The system receives first response information from the second terminal, the first response information including: a first response information in which the second terminal agrees to the sharing request;

[0096] The first response information and the terminal identification information of the second terminal are sent to the cloud device as the third information.

[0097] Optionally, the first information includes: second identification information;

[0098] The step of determining the corresponding second terminal based on the first information includes:

[0099] The corresponding second terminal is determined based on the second identification information.

[0100] Optionally, the first response information may further include at least one of the resolution and bandwidth corresponding to the second terminal.

[0101] Optional, also includes:

[0102] The second receiving module is used to receive the merging request and at least one second piece of information sent by the first terminal;

[0103] The fifth sending module is used to send fourth information in response to the merging request to the cloud device based on the second information;

[0104] The merging request is a merging request for at least a second portion of the screen image of the first terminal.

[0105] The fourth piece of information includes: the terminal identification information of the third terminal;

[0106] The third terminal is the terminal associated with the second information.

[0107] Optionally, sending fourth information regarding the merging request to the cloud device based on the second information includes:

[0108] Based on the second information, the corresponding third terminal is determined;

[0109] The merge request is sent to the third terminal;

[0110] Receive a second response from the third terminal indicating agreement to the merger request;

[0111] The second response information and the terminal identification information of the third terminal are sent to the cloud device as the fourth information.

[0112] Optionally, the second information includes: first identification information;

[0113] The step of determining the corresponding third terminal based on the second information includes:

[0114] Based on the first identification information, the corresponding third terminal is determined.

[0115] This invention also provides an information processing apparatus for use in cloud devices, comprising:

[0116] The third receiving module is used to receive third information sent by the server in response to a sharing request; the sharing request is a sharing request for at least a first portion of the screen of the first terminal; the third information includes: terminal identification information of the second terminal;

[0117] The first connection establishment module is used to establish a connection with the second terminal based on the terminal identification information of the second terminal.

[0118] The sixth sending module is used to send data concerning the at least first part of the screen image to the second terminal via the connection.

[0119] Optionally, the third information further includes: first response information; the first response information includes: a first response information indicating that the second terminal agrees to the sharing request;

[0120] The step of establishing a connection with the second terminal based on the terminal identification information of the second terminal includes:

[0121] A connection is established with the second terminal based on the terminal identification information and the first response information.

[0122] Optionally, the first response information may further include at least one of the resolution and bandwidth corresponding to the second terminal;

[0123] Sending data for at least the first portion of the screen display to the second terminal via the connection includes:

[0124] Based on the resolution and bandwidth, determine the maximum data transmission rate corresponding to the second terminal;

[0125] Based on the resolution, bandwidth, and maximum bitrate, data for at least the first portion of the screen image is sent to the second terminal via the connection.

[0126] Optionally, sending data regarding the at least first portion of the screen display to the second terminal via the connection includes:

[0127] Through the connection, the system receives operation instructions sent by the second terminal for at least the first portion of the screen display.

[0128] According to the operation instructions sent by the second terminal, data for the at least first part of the screen image is sent to the second terminal through the connection.

[0129] Optional, also includes:

[0130] The fourth receiving module is used to receive fourth information sent by the server in response to the merging request; the merging request is a merging request for at least the second part of the screen image of the first terminal; the fourth information includes: terminal identification information of the third terminal;

[0131] The second processing module is used to disconnect the connection with the third terminal based on the terminal identification information of the third terminal.

[0132] Optionally, the fourth information further includes: a second response information indicating that the third terminal agrees to the merging request;

[0133] Disconnecting from the third terminal based on the terminal identification information of the third terminal includes:

[0134] Based on the terminal identification information and the second response information of the third terminal, the connection with the third terminal is disconnected.

[0135] Optional, also includes:

[0136] The first determining module is used to determine first identification information based on the terminal identification information of the third terminal after disconnecting the connection with the third terminal based on the terminal identification information of the third terminal.

[0137] The second determining module is used to determine, based on the first identification information and the first mapping relationship, that the first control device is connected to the first terminal;

[0138] Wherein, the first mapping relationship is the mapping relationship between the first identification information and the first control device that controls the at least second part of the screen.

[0139] Optional, also includes:

[0140] The third processing module is used to process the operation instruction corresponding to the second information sent by the first terminal after determining that the first control device is connected to the first terminal according to the first identification information and the first mapping relationship; wherein the second information includes the first identification information; the second information is obtained by the first terminal according to the merging request.

[0141] This invention also provides an information processing device, including a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, it implements the aforementioned information processing method on the first terminal side; or,

[0142] When the processor executes the program, it implements the aforementioned server-side information processing method; or...

[0143] When the processor executes the program, it implements the aforementioned information processing method on the cloud device side.

[0144] This invention also provides a readable storage medium storing a program that, when executed by a processor, implements the steps of the information processing method on the first terminal side described above; or,

[0145] When this program is executed by the processor, it implements the steps in the aforementioned server-side information processing method; or...

[0146] When the program is executed by the processor, it implements the steps in the information processing method on the cloud device side described above.

[0147] The beneficial effects of the above-described technical solution of the present invention are as follows:

[0148] In the above solution, the information processing method obtains at least one first piece of information according to a sharing request; and sends the sharing request and the first piece of information to the server. The sharing request is for at least a first portion of the screen of the first terminal. This method supports the transformation of multi-user collaborative gaming on a single terminal into multi-user gaming on multiple terminals. For example, it allows multiple people in a family to play games collaboratively on the same TV, with some controllers transferred to computers, mobile phones, or other terminals in other rooms, enabling continued collaborative gaming in different rooms. This satisfies diverse user needs, facilitates user operation, and fully utilizes available terminals, effectively solving the problem that existing information processing solutions for multi-user cloud gaming fail to meet diverse user needs. Attached Figure Description

[0149] Figure 1 This is a schematic flowchart of the information processing method according to an embodiment of the present invention. Figure 1 ;

[0150] Figure 2 This is a schematic flowchart of the information processing method according to an embodiment of the present invention. Figure 2 ;

[0151] Figure 3 This is a schematic flowchart of the information processing method according to an embodiment of the present invention. Figure 3 ;

[0152] Figure 4 This is a schematic diagram illustrating multiple users operating a single game terminal according to an embodiment of the present invention.

[0153] Figure 5 This is a schematic diagram illustrating multiple game terminals operated by multiple users according to an embodiment of the present invention;

[0154] Figure 6 This is a schematic diagram of the game interface according to an embodiment of the present invention. Figure 1 ;

[0155] Figure 7 This is a schematic diagram of the specific application process of the information processing method according to an embodiment of the present invention. Figure 1 ;

[0156] Figure 8 This is a schematic diagram of the final screen sharing result in an embodiment of the present invention. Figure 1 ;

[0157] Figure 9 This is a schematic diagram of the final screen sharing result in an embodiment of the present invention. Figure 2 ;

[0158] Figure 10 This is a schematic diagram of the game interface according to an embodiment of the present invention. Figure 2 ;

[0159] Figure 11 This is a schematic diagram of the final screen merging result in an embodiment of the present invention. Figure 1 ;

[0160] Figure 12 This is a schematic diagram of the final screen merging result in an embodiment of the present invention. Figure 2 ;

[0161] Figure 13 This is a schematic diagram of the game interface according to an embodiment of the present invention. Figure 3 ;

[0162] Figure 14 This is a schematic diagram of the game interface according to an embodiment of the present invention. Figure 4 ;

[0163] Figure 15 This is a schematic diagram of the specific application process of the information processing method according to an embodiment of the present invention. Figure 2 ;

[0164] Figure 16 This is a schematic diagram of the game interface according to an embodiment of the present invention. Figure 5 ;

[0165] Figure 17 This is a schematic diagram of the information processing device structure according to an embodiment of the present invention. Figure 1 ;

[0166] Figure 18 This is a schematic diagram of the information processing device structure according to an embodiment of the present invention. Figure 2 ;

[0167] Figure 19 This is a schematic diagram of the information processing device structure according to an embodiment of the present invention. Figure 3 . Detailed Implementation

[0168] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0169] This invention addresses the problem that existing information processing solutions for multiplayer cloud games fail to meet the diverse needs of users, and provides an information processing method applied to a first terminal, such as... Figure 1 As shown, it includes:

[0170] Step 11: Obtain at least one piece of initial information based on the sharing request;

[0171] Step 12: Send the sharing request and the first information to the server; wherein the sharing request is a sharing request for at least a first portion of the screen of the first terminal.

[0172] The first information may be obtained from the second terminal. Specifically, the first information may be input by the first user into the second terminal, and may also be referred to as the first user's first information. The first information may include at least one of the first user's second identification information and a second mapping relationship. The second mapping relationship is a mapping relationship between the second identification information and the second control device that controls the at least first part of the screen. The "second identification information" may be determined by the second terminal (it may be sent by the second terminal to the first terminal and then forwarded to the server, and may also be sent to the cloud device, which is not limited here). Specifically, it can be understood as the first user's second identification information, but it is not limited thereto.

[0173] Furthermore, when "at least the first part of the screen" refers to the entire screen, the sharing request can be understood as a sharing request for the screen; when "at least the first part of the screen" refers to a part of the screen, the sharing request can be understood as a sharing request for the operation command; but it is not limited to these.

[0174] The information processing method provided in this embodiment of the invention obtains at least one first piece of information according to a sharing request; and sends the sharing request and the first piece of information to a server; wherein the sharing request is a sharing request for at least a first portion of the screen of the first terminal; it can support the transformation of multiple people operating one terminal to play games collaboratively, to multiple people operating multiple terminals to play games. For example, it can enable multiple people in a family to play games collaboratively through the same TV, and transfer the controllers of some people to computers, mobile phones, and other terminals in other rooms, so that they can continue to play games collaboratively in different rooms; thereby meeting the diverse needs of users, facilitating user use, making full use of the terminals around them, and effectively solving the problem that existing information processing solutions for multiplayer cloud games cannot meet the diverse needs of users.

[0175] Furthermore, after sending the sharing request and the first information to the server, the method further includes: stopping the transmission of the operation instructions corresponding to the received first information to the cloud device. This ensures that the cloud device executes the operation instructions corresponding to the first information quickly and accurately. The "operation instructions corresponding to the first information" can specifically be the operation instructions corresponding to the first user, but are not limited to this.

[0176] In this embodiment of the invention, the information processing method further includes: obtaining at least one second piece of information according to a merging request; sending the merging request and the second piece of information to the server; wherein the merging request is a merging request for at least a second portion of the screen image of the first terminal.

[0177] This allows for a shift from multiple users controlling multiple terminals to multiple users controlling a single control unit. The "second information" can be obtained from a third terminal; specifically, it can be input by the second user into the third terminal, also referred to as the second user's second information. The "second user" can belong to the aforementioned first user. The second information may include: first identification information, which can be determined by the third terminal (it can be sent from the third terminal to the first terminal, then forwarded to the server, and subsequently to cloud devices, without limitation). Specifically, it can be understood as the second user's first identification information, but is not limited to this. Furthermore, when "at least the second part of the screen display" refers to the entire screen, the merge request can be understood as a merge request for the entire screen; when "at least the second part of the screen display" refers to a portion of the screen, the merge request can be understood as a merge request for the operation instructions; however, it is not limited to this.

[0178] Furthermore, after sending the merge request and the second information to the server, the method further includes: sending the operation instruction corresponding to the received second information to the cloud device. This ensures that the cloud device can correctly execute the operation instruction corresponding to the second information. The "operation instruction corresponding to the second information" can specifically be the operation instruction corresponding to the second user, but is not limited to this.

[0179] This invention also provides an information processing method applied to a server, such as... Figure 2 As shown, it includes:

[0180] Step 21: Receive a sharing request and at least one first piece of information sent by the first terminal;

[0181] Step 22: Based on the first information, send third information to the cloud device in response to the sharing request; wherein the sharing request is a sharing request for at least a first portion of the screen of the first terminal; the third information includes: terminal identification information of the second terminal; the second terminal is the terminal associated with the first information.

[0182] Wherein, the first information may be obtained from the second terminal; specifically, the first information may be input by the first user into the second terminal, or may be referred to as the first user's first information; when "at least the first part of the screen" is the entire screen, the sharing request can be understood as a sharing request for the screen; when "at least the first part of the screen" is a part of the screen, the sharing request can be understood as a sharing request for the operation command; "the terminal associated with the first information" can be understood as the terminal associated with the first user; specifically, the second terminal is determined based on the first information; but is not limited thereto.

[0183] The information processing method provided in this embodiment of the invention receives a sharing request and at least one first piece of information sent by a first terminal; based on the first information, it sends third information to a cloud device in response to the sharing request; wherein the sharing request is a sharing request for at least a first portion of the screen of the first terminal; the third information includes: terminal identification information of a second terminal; the second terminal is a terminal associated with the first information; it can support the transformation of multiple people operating one terminal to play games collaboratively, to multiple people operating multiple terminals to play games. For example, it can enable multiple people in a family to play games collaboratively through the same TV, and transfer the controllers of some people to computers, mobile phones, and other terminals in other rooms, so that they can continue to play games collaboratively in different rooms; thereby meeting the diverse needs of users, facilitating user use, making full use of the terminals around them, and effectively solving the problem that existing information processing solutions for multiplayer cloud games cannot meet the diverse needs of users.

[0184] The step of sending third information to the cloud device in response to the sharing request based on the first information includes: determining the corresponding second terminal based on the first information; sending the sharing request to the second terminal; receiving first response information from the second terminal; and sending third information to the cloud device in response to the sharing request based on the first response information. The third information further includes the first response information, which includes a first response indicating that the second terminal agrees to the sharing request. This ensures that screen sharing occurs only when the second terminal agrees, improving the user experience.

[0185] In this embodiment of the invention, the first information includes: second identification information; determining the corresponding second terminal based on the first information includes: determining the corresponding second terminal based on the second identification information. This allows for accurate determination of the second terminal. The "second identification information" can be determined by the second terminal (it can be sent by the second terminal to the first terminal, then forwarded to the server, and subsequently sent to cloud devices, which is not limited here), specifically it can be understood as the second identification information of the first user, but is not limited thereto.

[0186] The first information may further include: a second mapping relationship; the second mapping relationship is a mapping relationship between the second identification information and the second control device that controls the at least first part of the screen image; the information processing method further includes: sending the second mapping relationship to the cloud device so that the cloud device can subsequently merge the screen images accordingly.

[0187] Furthermore, the first response information also includes at least one of the resolution and bandwidth corresponding to the second terminal. This enables the cloud device to perform appropriate data transmission accordingly.

[0188] In this embodiment of the invention, the information processing method further includes: receiving a merging request and at least one second piece of information sent by the first terminal; and sending fourth information regarding the merging request to a cloud device based on the second information; wherein the merging request is a merging request for at least a second portion of the screen display of the first terminal; and the fourth information includes: terminal identification information of a third terminal; the third terminal is the terminal associated with the second information. This enables a shift from multiple users controlling multiple terminals to multiple users controlling a single terminal.

[0189] The "second information" can be obtained from the third terminal. Specifically, the second information can be input by the second user into the third terminal, or it can be referred to as the second user's second information. When "at least the second part of the screen" is the entire screen, the merging request can be understood as a merging request for the screen. When "at least the second part of the screen" is a part of the screen, the merging request can be understood as a merging request for the operation command. The "terminal associated with the second information" can be understood as the terminal associated with the second user. Specifically, the third terminal is determined based on the second information, but it is not limited to this.

[0190] The step of sending fourth information regarding the merging request to the cloud device based on the second information includes: determining the corresponding third terminal based on the second information; sending the merging request to the third terminal; receiving second response information from the third terminal; and sending fourth information regarding the merging request to the cloud device based on the second response information. The fourth information further includes the second response information, which includes a second response indicating that the third terminal agrees to the merging request. This ensures that screen merging occurs only when the third terminal agrees, improving the user experience.

[0191] In this embodiment of the invention, the second information includes: first identification information; determining the corresponding third terminal based on the second information includes: determining the corresponding third terminal based on the first identification information. This allows for accurate determination of the third terminal. The "first identification information" can be determined by the third terminal (it can be sent from the third terminal to the first terminal and then forwarded to the server; subsequently, it can also be sent to cloud devices, which is not limited here), specifically it can be understood as the first identification information of the second user, but it is not limited thereto.

[0192] Furthermore, the information processing method may also include: the server monitoring the operating status of each terminal (including at least one of the first terminal, second terminal, and third terminal); in case of abnormal status (e.g., no response to operation), sending status report information to the cloud device; the cloud device automatically sharing or merging at least a portion of the screen display of the first terminal based on the status report information; for example, merging the screen display or related operation instructions of the terminal with abnormal status (the first terminal) onto other terminals (the second or third terminal); but not limited to this. This ensures that the solution can still continue to execute in abnormal situations.

[0193] This invention also provides an information processing method applied to cloud devices, such as... Figure 3 As shown, it includes:

[0194] Step 31: Receive third information from the server regarding the sharing request; the sharing request is a sharing request for at least a first portion of the screen of the first terminal; the third information includes: terminal identification information of the second terminal;

[0195] Step 32: Establish a connection with the second terminal based on the terminal identification information of the second terminal;

[0196] Step 33: Send data for at least the first part of the screen display to the second terminal via the connection.

[0197] In this embodiment, the second terminal is a terminal associated with the first user (or the first information). The embodiment may further include: after establishing a connection with the second terminal based on its terminal identification information, the following: not processing the previously received operation instructions corresponding to the first user (or the first information); wherein the first user is a user associated with the second terminal (or the second terminal is associated with the first information). This avoids executing meaningless instructions. Furthermore, when "at least the first part of the screen display" refers to the entire screen, the sharing request can be understood as a screen-specific sharing request; when "at least the first part of the screen display" refers to a portion of the screen, the sharing request can be understood as a sharing request for an operation instruction; however, this is not a limitation.

[0198] The information processing method provided in this embodiment of the invention receives third information from a server regarding a sharing request; the sharing request is a sharing request for at least a first portion of the screen of a first terminal; the third information includes: terminal identification information of a second terminal; a connection is established with the second terminal based on the terminal identification information of the second terminal; and data for the at least first portion of the screen is sent to the second terminal through the connection. This method supports the transformation of multi-user collaborative gaming on a single terminal into multi-user gaming on multiple terminals. For example, it allows multiple people in a family to play games collaboratively on the same television, with some controllers transferred to computers, mobile phones, or other terminals in other rooms, enabling continued collaborative gaming in different rooms. This satisfies diverse user needs, facilitates user operation, and fully utilizes available terminals, effectively solving the problem that existing information processing solutions for multi-user cloud gaming fail to meet diverse user needs.

[0199] Furthermore, the third information also includes: first response information; the first response information includes: a first response information indicating that the second terminal agrees to the sharing request; establishing a connection with the second terminal based on the terminal identification information of the second terminal includes: establishing a connection with the second terminal based on the terminal identification information of the second terminal and the first response information. This ensures that screen sharing occurs only when the second terminal agrees to share, improving the user experience.

[0200] In this embodiment of the invention, the first response information further includes at least one of the resolution and bandwidth corresponding to the second terminal; sending data for the at least first portion of the screen image to the second terminal through the connection includes: determining the maximum bitrate of data transmission corresponding to the second terminal based on the resolution and bandwidth; and sending data for the at least first portion of the screen image to the second terminal through the connection based on the resolution, bandwidth, and maximum bitrate. This can improve the adaptability of data transmission. Specifically, it may include the following:

[0201] (1) In this embodiment of the invention, the first response information further includes: data transmission parameter information of the second terminal; the step of sending data for the at least first part of the screen image to the second terminal through the connection includes: sending data for the at least first part of the screen image to the second terminal through the connection according to the data transmission parameter information of the second terminal. This can improve the adaptability of data transmission.

[0202] (2) Wherein, the data transmission parameter information of the second terminal includes at least one of the resolution and bandwidth corresponding to the second terminal; the step of sending data for the at least first part of the screen image to the second terminal through the connection according to the data transmission parameter information of the second terminal includes: determining the maximum bit rate of data transmission corresponding to the second terminal according to the resolution and bandwidth; and sending data for the at least first part of the screen image to the second terminal through the connection according to the resolution, bandwidth and maximum bit rate. This can further ensure the adaptability of data transmission.

[0203] In this embodiment of the invention, sending data for the at least first portion of the screen display to the second terminal via the connection includes: receiving an operation instruction for the at least first portion of the screen display sent by the second terminal via the connection; and sending data for the at least first portion of the screen display to the second terminal via the connection according to the operation instruction sent by the second terminal. This enables accurate data transmission.

[0204] Furthermore, the information processing method further includes: receiving fourth information sent by the server in response to a merging request; the merging request is a merging request for at least a second portion of the screen display of the first terminal; the fourth information includes: terminal identification information of a third terminal; and disconnecting the connection with the third terminal based on the terminal identification information of the third terminal. This enables a shift from multiple users controlling multiple terminals to multiple users controlling a single terminal.

[0205] Furthermore, when "at least the second part of the screen display" refers to the entire screen, the merging request can be understood as a merging request for the entire screen; when "at least the second part of the screen display" refers to a partial screen, the merging request can be understood as a merging request for the operation instructions; however, it is not limited to these limitations. Specifically, "disconnecting from the third terminal based on the terminal identification information of the third terminal" can include: disconnecting from the third terminal based on the terminal identification information of the third terminal, ceasing to receive operation instructions sent by the third terminal, and ceasing to transmit data to the third terminal.

[0206] In this embodiment of the invention, the fourth information further includes: second response information; the second response information includes: a second response information indicating that the third terminal agrees to the merging request; the step of disconnecting the connection with the third terminal based on the terminal identification information of the third terminal includes: disconnecting the connection with the third terminal based on the terminal identification information of the third terminal and the second response information. This ensures that screen merging is performed when the third terminal agrees to the merging, improving the user experience.

[0207] Furthermore, after disconnecting from the third terminal based on its terminal identification information, the method further includes: determining first identification information based on the third terminal's terminal identification information; and determining that a first control device is connected to the first terminal based on the first identification information and a first mapping relationship. The first mapping relationship is the mapping relationship between the first identification information and the first control device that controls at least the second portion of the screen display. This ensures the normal execution of subsequent user commands after the screen display is merged.

[0208] Specifically, "determining the first identification information" may include: determining the first identification information of the second user associated with the third terminal; the "first identification information" may be determined by the third terminal (it may be sent by the third terminal to the first terminal and then forwarded to the server, and may also be sent to the cloud device, which is not limited here), and may be understood as the first identification information of the second user, but is not limited thereto.

[0209] Furthermore, the "first mapping relationship" may be included in the aforementioned second mapping relationship, but is not limited thereto; furthermore, before determining the connection between the first control device and the first terminal based on the first identification information and the first mapping relationship, the process may further include: receiving the second mapping relationship sent by the server; storing the second mapping relationship; wherein, the second mapping relationship is the mapping relationship between the second identification information of the first user and the second control device controlling the at least first part of the screen display. The "second identification information" may be determined by the second terminal (it may be sent by the second terminal to the first terminal and then forwarded to the server, and subsequently sent to the cloud device, which is not limited here), specifically it can be understood as the second identification information of the first user, but is not limited thereto. The first user may include the second user, and the second control device may include the first control device, but is not limited thereto.

[0210] In this embodiment of the invention, after determining that the first control device is connected to the first terminal based on the first identification information and the first mapping relationship, the method further includes: processing an operation instruction corresponding to second information sent by the first terminal; wherein the second information includes the first identification information; and the second information is obtained by the first terminal according to the merging request. This ensures the normal execution of the operation instruction corresponding to the second information. Specifically, the "operation instruction corresponding to the second information" can be understood as the operation instruction corresponding to the second user.

[0211] The information processing method provided in the embodiments of the present invention will be illustrated below with examples.

[0212] To address the aforementioned technical problems, embodiments of the present invention provide an information processing method, specifically a scheme for sharing and merging game screens and operation commands; including the following points:

[0213] (1) This application proposes a screen sharing scheme that can share the screen of a game terminal (corresponding to the first terminal mentioned above) to multiple terminals (corresponding to the second terminal mentioned above). The cloud gaming instance (corresponding to the cloud device mentioned above) sends the same game screen to the terminals with different resolutions and bitrates according to the resolution, network bandwidth, etc. of different terminals, making full use of the terminals and realizing distributed gaming.

[0214] (2) The present invention also proposes a screen merging scheme, which is the opposite of screen sharing.

[0215] (3) The present invention also proposes a scheme for sharing operation instructions, which can distribute the game character that was originally operated by one person to multiple people to operate at the same time.

[0216] (4) The present invention also proposes a scheme for merging operation instructions, which is the opposite of operation instruction sharing.

[0217] (5) The embodiments of the present invention also propose that, in the scenario of sharing operation commands, screen sharing and / or operation command merging can be automatically initiated according to possible abnormal situations.

[0218] In this embodiment of the invention, multiple players can jointly operate a single game terminal during multiplayer gameplay, such as... Figure 4 As shown, multiple users can also operate multiple terminals, such as... Figure 5 As shown; based on this, embodiments of the present invention propose a conversion scheme for these two multiplayer game modes, including screen sharing ( Figure 4 Switch to Figure 5 ) and screen merging ( Figure 5 Switch to Figure 4Specifically, this can include: when multiple users operate a game terminal simultaneously through different controllers, they can request the IDs of other users and the mapping relationships between each user and their respective controllers from the server. The server can then share the game interface of this terminal with the corresponding terminals of other users, enabling multiple users to operate different terminals through different controllers. Alternatively, when multiple users operate different terminals through different controllers, one terminal can send a request to the server to merge screens, along with the terminal of the user requesting the screen merge. This allows the game interfaces of multiple terminals to be merged onto one terminal, enabling multiple users to operate a single terminal simultaneously through different controllers. Through these methods, this embodiment of the invention can achieve mutual conversion between two game modes: multiple users operating a single terminal simultaneously through different controllers and multiple users operating different terminals through different controllers. Users can choose the game mode according to the number of terminals and their needs, enhancing the gaming experience.

[0219] The following provides specific examples of screen sharing and screen merging.

[0220] 1. Screen sharing ( Figure 4 Switch to Figure 5 This corresponds to the case where at least the first part of the screen is the entire screen.

[0221] In this scenario, the screen is shared to three gaming terminals (users A, B, and C each operate a gaming terminal, designated T1, T2, and T3 respectively, with sharing initiated by T1). Assume that user A is logged into T1 (corresponding to the first terminal mentioned above):

[0222] Step 1: As Figure 6 As shown, user A clicks the share screen button b on the game interface a of game terminal T1, which pops up an input box c. In input box c, user A enters the unique IDs of user B and user C (corresponding to the second identification information in the first information above). The user can click "+" to add more input boxes c. Each input box c contains one unique user ID, and then user A clicks "OK". After clicking "OK", user A quickly and consecutively enters two operation commands on the corresponding controller (corresponding to the second control device above) (the two commands can be the same and the interval cannot exceed 0.5 seconds), forming a mapping relationship between the controller and the user. For example, after entering user B's user ID and clicking "OK", user A quickly presses any button on the controller currently used by user B twice.

[0223] In this example, you need to click the plus sign and enter the unique IDs of B and C in the two input boxes respectively. After entering the information, click "OK".

[0224] Step 2: Game terminal T1 sends a screen sharing request (corresponding to the sharing request) and the unique IDs of B and C to the coordination server (corresponding to the server mentioned above), as well as the mapping relationship between B and C and the controller (corresponding to the second mapping relationship mentioned above).

[0225] Step 3: The coordination server determines the corresponding game terminals T2 and T3 (corresponding to the second terminal mentioned above) based on the unique identifiers of B and C, and sends screen sharing information (corresponding to the sharing request mentioned above) to T2 and T3 respectively; users B and C click to accept the sharing on their respective terminals T2 and T3, and transmit the acceptance result (corresponding to the first response information mentioned above) and their respective terminal information (corresponding to the data transmission parameter information mentioned above, which may include resolution, bandwidth, etc.) to the coordination server. In this embodiment of the invention, the mapping relationship between the controller and terminals T2 and T3 can also be obtained according to the mapping relationship, but it is not limited thereto.

[0226] Step 4: The coordinating server sends information to the cloud gaming instance (corresponding to the cloud device mentioned above) indicating that users B and C agree to share their screens (corresponding to the first response information mentioned above), as well as B and C's respective terminal information (corresponding to the data transmission parameter information mentioned above, which may include resolution, bandwidth, etc.) and terminal identification information (corresponding to the terminal identification information of the second terminal mentioned above) to the cloud gaming instance (corresponding to the second terminal's terminal identification information).

[0227] Step 5: Establish a connection between the cloud gaming instance and gaming terminals T2 and T3 (corresponding to the connection established with the second terminal based on the terminal identification information of the second terminal). The resolution of the corresponding video stream can be set to R2 and R3 according to the resolutions (R2 and R3) of T2 and T3. The maximum bitrate of the video stream (corresponding to the maximum bitrate of the data transmission mentioned above) can be set according to the bandwidths (B2 and B3, in Kbps) of T2 and T3. and It also receives operation instructions from users B and C transmitted via T2 and T3 (corresponding to the operation instructions sent by the second terminal for at least the first part of the screen image via the connection mentioned above). Furthermore, T1 does not transmit the operation instructions from the controller previously used by B and C, and the cloud gaming instance does not process the operation instructions from the controller previously used by B and C (i.e., the client does not transmit, and the server does not process); this corresponds to the first terminal ceasing to send the operation instructions corresponding to the received first information (corresponding to the first user) to the cloud device; the cloud device not processing the operation instructions corresponding to the previously received first information; wherein, the first information is associated with the second terminal (which can also be understood as the first user being the user associated with the second terminal). For example:

[0228] Assume that before screen sharing, the commands for moving characters upwards using the three controllers of T1 are UP_1P, UP_2P, and UP_3P (the suffixes 1P, 2P, and 3P represent the three characters controlled by the user in the game). Furthermore, in the first step, B is already associated with UP_2P, and C with UP_3P. After screen sharing, T1's operation commands are UP_1P, UP_2P(NO), and UP_3P(NO), where NO indicates that the client does not transmit operation commands and the server does not process them. T2's operation command is UP_2P_S_BID, and T3's operation command is UP_3P_S_CID, where S represents sharing the operation command, and BID and CID are the user IDs of BC (i.e., users B and C), respectively. After receiving UP_2P_S_BID and UP_3P_S_CID, the cloud gaming instance performs security checks on the BID and CID to check if they match the user ID, and processes UP_2P_S_BID into UP_2P and UP_3P_S_CID into UP_3P, and applies them to the game.

[0229] For the screen sharing process, please refer to [link / reference]. Figure 7 The final result obtained can be as follows Figure 8 As shown; however, if only B's user ID is entered in the second step, or only B accepts in the fourth step, then the final sharing result will only share the screen to T2, as shown. Figure 9 As shown; the same applies to user C, so I will not elaborate further.

[0230] Based on the above, in this embodiment of the invention, multiple terminals control the same game based on the relationship between the controller and the terminal. Simultaneously, based on the different resolutions and network bandwidths of the terminals, the same game screen is sent to the terminals at different resolutions and bitrates. Specifically, this includes the following:

[0231] The initial scenario involves multiple users operating multiple controllers to play the game through a single terminal. In other words, multiple people use multiple controllers on the same terminal to control different characters in the same game. To address this, the controllers are moved to multiple terminals. Previously, users A, B, C, and D were using four controllers on terminal A to control different characters in the same game. Now, B's controller is moved to terminal B, C's to terminal C, and D's to terminal D. This allows users to simultaneously control different characters in the same game, enabling multiple users to play the same game.

[0232] The cloud gaming instance can obtain the mapping relationship between new terminals and controllers, establish connections with each new terminal, and set the resolution and bandwidth of the corresponding video stream according to the resolution and bandwidth of each new terminal. Based on this, it pushes the cloud gaming video stream to the corresponding terminal. When receiving (operation) instructions, the new terminal transmits the instruction stream of the corresponding controller, while the original terminal no longer transmits the instructions of the transferred controller. The cloud gaming instance also no longer processes the instructions sent by the transferred controller on the original terminal based on the relationship between the terminal and the controller.

[0233] Specifically, in the screen merging solution, the operation instructions are independent (in this case, it is not about turning one terminal into multiple terminals, but about distributing the operation instructions on one terminal to multiple terminals).

[0234] In this embodiment of the invention, the screen merging scheme described above allows multiple people in a household to collaboratively play games on the same television. Afterward, some participants can transfer their controllers to computers, mobile phones, or other terminals in other rooms, enabling them to continue playing games collaboratively in different rooms. Furthermore, it addresses the need for multiple terminals (computers, mobile phones, etc.) to then transfer their controllers back to the same television for collaborative gaming. This merging and separation of terminals has practical application requirements, and this embodiment of the invention has promising application prospects.

[0235] 2. Screen merging ( Figure 5 Switch to Figure 4 This corresponds to the case where at least the second part of the screen is the entire screen.

[0236] Screen merging is the opposite of screen sharing. For example, merging T3 and T2 into T1. Specifically, the screen merging process may include the following steps:

[0237] Step 1: As Figure 10 As shown, in the game interface d of game terminal T1 (corresponding to the first terminal mentioned above), click merge screen (button e), and a list f of user IDs that have shared the screen will pop up. Multiple selections can be made in list f. Click OK.

[0238] In this example, you need to select the unique IDs of B and C in list f, and then click OK after making the selection.

[0239] Step 2: Game terminal T1 sends a request to merge screens (corresponding to the above-mentioned merge request) and the unique IDs of B and C (corresponding to the first identification information in the second information) to the coordination server (corresponding to the above-mentioned server).

[0240] Step 3: The coordination server determines the corresponding game terminals T2 and T3 (corresponding to the third terminal mentioned above) based on the unique identifiers of B and C, and sends screen merging information (corresponding to the merge request mentioned above) to T2 and T3 respectively. Users B and C click to accept the merge on their respective terminals T2 and T3, and then transmit the acceptance result (corresponding to the second response information mentioned above) to the coordination server.

[0241] Step 4: The coordination server sends information to the cloud gaming instance (corresponding to the cloud device) indicating that users B and C agree to merge their screens (corresponding to the fourth information mentioned above), such as including the identifiers of T2 and T3 (corresponding to the terminal identifier information of the third terminal mentioned above).

[0242] Step 5: The cloud gaming instance disconnects from gaming terminals T2 and T3 (corresponding to the above-mentioned disconnection from the third terminal based on the terminal identification information of the third terminal), ceases transmitting audio and video streams to T2 and T3 (corresponding to the above-mentioned ceasing of data transmission to the third terminal), and ceases receiving operation commands transmitted by T2 and T3 (corresponding to the above-mentioned ceasing of receiving operation commands sent by the third terminal). Then, based on the mapping relationship received during screen sharing (corresponding to the above-mentioned first mapping relationship), the controller (corresponding to the above-mentioned first control device) connected to T2 and T3 is reconnected to T1. Then, the operation commands that were not transmitted in T1 during screen sharing are resumed (corresponding to the above-mentioned operation commands corresponding to the second information (corresponding to the second user) received by the first terminal and sent to the cloud device; that is, the commands for users B and C that T1 is now transmitting normally). The cloud gaming instance resumes processing operation commands that were not previously processed (corresponding to the above-mentioned operation commands corresponding to the second information sent by the first terminal that the cloud device is processing; that is, the commands for users B and C transmitted by T1 that the cloud instance is now processing normally).

[0243] The final result of screen merging can be as follows: Figure 11 As shown; however, if only user ID C is selected in the first step, or only C accepts in the third step, then the final merge result will only merge T3 into T1, while T2 will remain, as shown. Figure 12 As shown; the same applies to user B, so I will not elaborate further.

[0244] Furthermore, the solution provided in the embodiments of the present invention can also be implemented as sharing operation instructions or merging operation instructions, as illustrated in the following specific examples.

[0245] 3. Sharing of operation instructions, corresponding to the situation where at least the first part of the screen is a partial screen;

[0246] In this scenario, the actions of a character can be assigned to multiple different users, with each user controlling a part of the character's actions; for example, a game character can move and fire bullets, with user A responsible for moving the character and user B responsible for firing bullets, which can break through the game's limitation on the number of players.

[0247] User A uses game terminal T1 (corresponding to the first terminal mentioned above). T1 shares operations with T2 and T3 (corresponding to the second terminal mentioned above). Users B and C use game terminals T2 and T3 respectively (the terminal here may include a controller, or be used as a controller, the controller corresponding to the second control device mentioned above).

[0248] Step 1: As Figure 13 and 14 As shown, clicking the share command button h (in the game interface g) on ​​the T1 screen will bring up a pop-up layer i;

[0249] Step 2: In the input box after (overlay i) To (item), enter the user IDs of B and C (corresponding to the second identifier information in the first information above). The From field is fixed to A's ID (A is logged in on T1, so the command sharing initiated on T1 can only be initiated by A). Clicking "+" increases the number of people sharing (the "+" can be hidden after the user addition is complete; this is not limited here). Then (as shown in the image)... Figure 13 and 14 (As shown) Click the "Select Operation Area" button, then, as... Figure 14 As shown, the operation areas for A, B, and C are selected respectively (k, m, and n are the operation areas for A, B, and C respectively).

[0250] The "Select Operation Area" button j for user A can be displayed after the user addition process is complete, or it can be displayed before the user addition process is complete; there is no limitation on this. Furthermore, the method for selecting the operation area can be the same as the method for determining the mapping relationship when sharing the screen, such as selecting by quickly pressing the same command twice; for example:

[0251] After clicking the "Select Operation Area" button j to the right of A, press the up, down, left, and right arrow keys to select the operation area k of A. Similarly, click the "Select Area" button to the right of B and C to select the operation area of ​​B and C respectively.

[0252] Step 3: Send the user IDs of A, B, and C and their respective selected operating regions (which may be included in the above sharing request) to the coordination server (corresponding to the above server). The coordination server determines the corresponding game terminals T2 and T3 (corresponding to the above second terminal) based on the unique identifiers of B and C, and forwards the relevant information (such as the user IDs of A, B, and C and their respective selected operating regions) and the identifiers of T2 and T3 (corresponding to the terminal identifier information of the above second terminal) to the cloud gaming instance (corresponding to the above cloud device).

[0253] Step 4: The cloud gaming instance establishes a connection with T2 and T3 (corresponding to the above-mentioned connection establishment with the second terminal based on the terminal identification information of the second terminal), and sends a "whether to agree to the sharing operation".

[0254] This is to clarify that the cloud gaming instance sends a sharing request to T2 and T3 (and subsequently receives response information), but this is not a limitation; for example, the coordination server could also send a sharing request to T2 and T3, receive response information (corresponding to the first response information mentioned above), and then send it to the cloud gaming instance.

[0255] Step 5: After B and C agree (i.e., receive the response message), the cloud gaming instance sends the operation areas previously selected by A, B, and C to T1, T2, and T3 respectively. Each terminal, based on the area selected in Step 3, hides or masks the areas it did not select (the client does not send, and the server does not process; see the relevant content on screen sharing, which will not be repeated here). If only B agrees in Step 5, the cloud gaming instance sends the area selected by B to T2, and T1 only hides or masks the area operated by B. The same applies if only C agrees, which will not be repeated here.

[0256] In addition, the coordinating server can also continuously monitor the status of T1, T2, and T3.

[0257] Based on the above, the detailed process for sharing operation instructions can be found in [link / reference]. Figure 15 .

[0258] 4. Operation instructions are merged, corresponding to the situation where at least the second part of the screen is a partial screen;

[0259] Merging and sharing operation instructions are opposite operations. For example, merging operation instructions T2 and T3 into T1. Specifically, the operation instruction merging process may include the following steps:

[0260] Step 1: As Figure 16 As shown, in the game interface p of game terminal T1 (corresponding to the first terminal mentioned above), clicking the merge operation command button q will pop up a pop-up layer r, which lists the results of the previous sharing operation by user A.

[0261] Step 2: As Figure 16 As shown, you can click "-" on T1 to reduce the number of items to be merged. For example, to merge only B, you can click "-" to the right of C to delete C.

[0262] Step 3: Game terminal T1 sends the result (such as the second identification information in the above-mentioned merge request and second information) to the coordination server (corresponding to the above-mentioned server). The coordination server determines the corresponding game terminals T2 and T3 (corresponding to the above-mentioned third terminal) based on the unique identifiers of B and C (corresponding to the above-mentioned first identification information), and forwards the relevant information (such as the user IDs of B and C and their respective corresponding operating areas) and the identifiers of T2 and T3 (corresponding to the terminal identification information of the above-mentioned second terminal) to the cloud gaming instance (corresponding to the above-mentioned cloud device).

[0263] Step 4: Based on the received information, the cloud gaming instance sends the merge instruction information to T1, T2, and T3.

[0264] Step 5: T1 displays all operable areas, T2 and T3 exit the operation interface, disconnect from the cloud gaming instance, and return to the cloud gaming APP homepage.

[0265] In addition, other relevant information is explained below:

[0266] 1. The embodiments of the present invention can restrict the above-mentioned instruction merging to occur after instruction sharing, and (merging and sharing) must be initiated by the same terminal, and instruction merging can only be accepted and not rejected; that is, after T1 shares the operation instructions with T2 and T3, T1, as the instruction sharing initiator, can only initiate instruction merging, while T2 and T3 can only accept. T1 can delete some terminals during instruction merging and only merge the instructions of some terminals.

[0267] Furthermore, in this embodiment of the invention, the operation command merging can also be performed as follows: the cloud gaming instance sends a merging request and other information to T2 and T3 based on the received information (and subsequently receives response information), and then disconnects from T2 and T3 (corresponding to the above-mentioned disconnection from the third terminal based on the terminal identification information of the third terminal). Afterwards, the cloud gaming instance can determine that users B and C no longer have operation permissions; or, based on the relevant information received during the previous command sharing, it can determine that the controllers connected to T2 and T3 reconnect to T1. For details, please refer to the relevant content on screen merging, which will not be repeated here.

[0268] Alternatively, in this embodiment of the invention, the coordination server can send a merge request to T2 and T3, and receive response information (corresponding to the second response information mentioned above) and send it to the cloud gaming instance. Based on the received information, the cloud gaming instance disconnects from T2 and T3 (corresponding to disconnecting from the third terminal based on the terminal identification information of the third terminal mentioned above). Afterwards, the cloud gaming instance can determine that users B and C no longer have operating permissions; or, based on the relevant information received during the previous instruction sharing, it can determine that the controllers connected to T2 and T3 reconnect to T1. For details, please refer to the relevant content on screen merging, which will not be repeated here.

[0269] 2: Command sharing only shares the operation area, not the screen. In the scenario described above, T2 and T3 can be restricted to only display the selected operation area and not the game screen. That is, T2 and T3 are only used as controllers.

[0270] 3. Command sharing can be used simultaneously with screen sharing. For example, A, B, and C use T1, T2, and T3 respectively. T1 shares its screen with T2, and at the same time, T1 shares some operation commands with T3. At this time, T1 and T2 display the game screen and the corresponding operation area (T1's operable area and all of T2's operable areas), while T3 does not display the game screen, but only displays T3's operable area.

[0271] 4. During command sharing, the coordination server can also monitor the status of each terminal. When a terminal becomes unresponsive due to disconnection or other reasons, the coordination server sends the status to the cloud gaming instance. The cloud gaming instance automatically merges the operation commands from the unresponsive terminals based on the command sharing information and automatically shares the screen as needed. For example:

[0272] A, B, and C use T1, T2, and T3 respectively. T1 shares operation commands with T2 and T3. After a period of time, if the coordination server detects that T2 is unresponsive, it can send the unresponsiveness information of T2 to the cloud gaming instance. The cloud gaming instance then executes the operation command to merge T2. If T1 is unresponsive, since T1 is the initiator of command sharing and does not share the screen, T2 and T3, as controllers, do not display the game screen. In this case, the coordination server can send status information to the cloud gaming instance, which automatically performs a screen sharing operation, sharing the screen with the one with lower latency between T2 and T3 (let's say T2), and merging T1's operation area into T2. T2 then becomes the new initiator of command sharing. At this point, T1 is unresponsive, T2 displays the game screen and the operation areas of T1 and T2, and T3 displays T3's operation area. T2 becomes the new initiator of command sharing, and subsequently, it can be restricted that only T2 can initiate command merging operations.

[0273] The following are specific application examples of screen sharing and merging, as well as command sharing and merging:

[0274] Suppose three users, A, B, and C, are playing a game together on the TV in the living room. The living room is small, and the three people are crowded together. However, there is an empty computer in the study. In this case, C can open a cloud gaming app on the computer and log in. Then, C can share the TV screen to the computer and enter C's user ID. In this way, C can play with A and B on the computer in the study.

[0275] Suppose B needs to use the restroom later, but in order to continue playing, the screen can be shared to B's phone. B can then open the cloud gaming app on their phone and log in using their user ID. This way, A can play on the TV in the living room, B can play on their phone in the restroom, and C can play on their computer in the study. The three people can play the same game on three different devices.

[0276] Suppose that B comes out of the bathroom and wants to continue playing games on the TV. At this point, B's screen can be merged onto the TV screen. Meanwhile, A and B are playing on the TV in the living room, and C is playing on the computer in the study.

[0277] Suppose a new user, D, joins the game. Since the game can only be played by a maximum of three people, A can transfer some control permissions (corresponding to shared control commands) to D. For example, A can control the character's movement, while D can control all other character actions. At this point, A and B play on the TV in the living room, and D also plays on the TV in the living room, but using a separate terminal as the controller. C plays on the computer in the study.

[0278] It should be noted that the above scheme can restrict users to opening and logging into the cloud gaming app when using the terminal, and one account can only log in to one terminal at a time.

[0279] Therefore, the solution provided by the embodiments of the present invention includes the following parts:

[0280] 1) Screen sharing solutions, which may include the following:

[0281] Sharing the game interface of one terminal to other terminals involves the server obtaining the ID and terminal information of the user to be shared, binding the user ID to the controller, and then sending the terminal information of each user to the cloud gaming instance. This establishes a connection between the cloud gaming instance and the terminal of the user to be shared, and transmits the game's audio and video streams to the terminal of the user to be shared. In other words, the game interface of one terminal can be shared to other terminals, and multiple users can either operate the game through one terminal or operate the game through multiple terminals separately.

[0282] 2) The screen merging solution may include the following:

[0283] This process merges the game interfaces of multiple terminals into one terminal. Specifically, the server obtains the ID and terminal information of the user to be merged. After the user agrees to the merge, the cloud server instance disconnects from the terminal corresponding to the user to be merged. The user then performs game operations through the merged terminal. In other words, it merges the game interfaces that were originally operated by multiple users through different controllers onto one terminal. It is a dynamic transformation of the game mode, where multiple users can operate the same terminal or different terminals.

[0284] 3) A scheme for sharing operation instructions, which may include the following:

[0285] Multiple operation commands for a single character will be assigned to terminals corresponding to multiple different users. Specifically, the server can obtain the ID of each user, the corresponding operation area and terminal information of each user, and send the ID, operation information and terminal information of each user to the cloud gaming instance. The cloud gaming instance then sends each operation area to the corresponding terminal.

[0286] 4) A scheme for merging operation instructions, which may specifically include the following:

[0287] To merge operation commands from multiple terminals into a single terminal, the server can obtain the user ID of the operation command to be merged and the terminal information corresponding to each user ID. The server then sends the ID of the user to be merged and the corresponding terminal information to the cloud gaming instance. Based on the received information, the cloud gaming instance sends the merge command information to the terminal corresponding to each user, thereby controlling the terminal to be merged to exit the game operation interface.

[0288] 5) Abnormal handling method: When sharing operation commands, if a terminal does not respond, the operation commands of the unresponsive terminal will be automatically merged, and screen sharing can be automatically initiated as appropriate.

[0289] In summary, this invention addresses the challenges of existing multiplayer cloud gaming solutions (non-online games) where multiple players operate on the same terminal, making it difficult to fully utilize available terminals, and requiring all players to be near the same gaming terminal. The invention proposes the following solutions:

[0290] (1) A screen sharing and merging scheme that allows the screen of one terminal to be shared across multiple terminals during multiplayer cloud gaming (non-online games), with each player operating a separate terminal, thus achieving distributed gaming. All players in a multiplayer game can be located in different rooms, or even in multiple cities. Specifically, based on cloud gaming, the system can switch between multiple users operating the same terminal and multiple users operating different terminals according to user needs.

[0291] (2) A scheme for sharing and merging operation instructions can break through the game's limitation on the number of players, allowing multiple players to operate a game character at the same time.

[0292] (3) When sharing operation instructions, automatically perform screen sharing and / or merge operation instructions according to possible abnormal situations.

[0293] This invention also provides an information processing device applied to a first terminal, such as... Figure 17 As shown, it includes:

[0294] The first acquisition module 171 is used to acquire at least one piece of first information based on the sharing request;

[0295] The first sending module 172 is used to send the sharing request and the first information to the server;

[0296] The sharing request is a sharing request for at least a first portion of the screen of the first terminal.

[0297] The information processing device provided in this embodiment of the invention obtains at least one first piece of information according to a sharing request; and sends the sharing request and the first piece of information to a server; wherein the sharing request is a sharing request for at least a first portion of the screen image of the first terminal; it can support the transformation of multiple people operating one terminal to play games collaboratively, to multiple people operating multiple terminals to play games. For example, it can enable multiple people in a family to play games collaboratively through the same TV, and transfer the controllers of some people to computers, mobile phones, and other terminals in other rooms, so that they can continue to play games collaboratively in different rooms; thereby meeting the diverse needs of users, facilitating user use, making full use of the terminals around them, and effectively solving the problem that existing information processing solutions for multiplayer cloud games cannot meet the diverse needs of users.

[0298] Furthermore, the information processing device further includes: a first processing module, configured to stop sending the operation instructions corresponding to the received first information to the cloud device after sending the sharing request and the first information to the server.

[0299] In this embodiment of the invention, the information processing device further includes: a second acquisition module, configured to acquire at least one second piece of information according to a merging request; and a second sending module, configured to send the merging request and the second piece of information to the server; wherein the merging request is a merging request for at least a second portion of the screen image of the first terminal.

[0300] Furthermore, the information processing device further includes a third sending module, used to send an operation instruction corresponding to the received second information to the cloud device after sending the merging request and the second information to the server.

[0301] The aforementioned implementation embodiments of the information processing method on the first terminal side are all applicable to the embodiments of the information processing device and can achieve the same technical effect.

[0302] This invention also provides an information processing device applied to a server, such as... Figure 18 As shown, it includes:

[0303] The first receiving module 181 is used to receive a sharing request and at least one first piece of information sent by the first terminal;

[0304] The fourth sending module 182 is used to send third information in response to the sharing request to the cloud device based on the first information;

[0305] The sharing request is a sharing request for at least a first portion of the screen of the first terminal.

[0306] The third information includes: the terminal identification information of the second terminal;

[0307] The second terminal is the terminal associated with the first information.

[0308] The information processing device provided in this embodiment of the invention receives a sharing request and at least one first piece of information sent by a first terminal; based on the first information, it sends third information to a cloud device in response to the sharing request; wherein the sharing request is a sharing request for at least a first portion of the screen of the first terminal; the third information includes: terminal identification information of a second terminal; the second terminal is a terminal associated with the first information; it can support the transformation of multiple people operating one terminal to play games collaboratively, to multiple people operating multiple terminals to play games. For example, it can enable multiple people in a family to play games collaboratively through the same TV, and transfer the controllers of some people to computers, mobile phones, and other terminals in other rooms, so that they can continue to play games collaboratively in different rooms; thereby meeting the diverse needs of users, facilitating user use, making full use of the terminals around them, and effectively solving the problem that existing information processing solutions for multiplayer cloud games cannot meet the diverse needs of users.

[0309] The step of sending third information to the cloud device in response to the sharing request based on the first information includes: determining the corresponding second terminal based on the first information; sending the sharing request to the second terminal; receiving first response information from the second terminal; and sending third information to the cloud device in response to the sharing request based on the first response information. The third information further includes the first response information, which includes a first response from the second terminal agreeing to the sharing request.

[0310] In this embodiment of the invention, the first information includes: second identification information; determining the corresponding second terminal based on the first information includes: determining the corresponding second terminal based on the second identification information.

[0311] Furthermore, the first response information also includes at least one of the resolution and bandwidth corresponding to the second terminal.

[0312] In this embodiment of the invention, the information processing device further includes: a second receiving module, configured to receive a merging request and at least one second piece of information sent by the first terminal; and a fifth sending module, configured to send fourth information in response to the merging request to a cloud device based on the second information; wherein the merging request is a merging request for at least a second portion of the screen image of the first terminal; and the fourth information includes: terminal identification information of a third terminal; the third terminal is the terminal associated with the second information.

[0313] The step of sending fourth information to the cloud device in response to the merging request based on the second information includes: determining the corresponding third terminal based on the second information; sending the merging request to the third terminal; receiving second response information from the third terminal; and sending fourth information to the cloud device in response to the merging request based on the second response information. The fourth information further includes the second response information, which includes a second response from the third terminal agreeing to the merging request.

[0314] In this embodiment of the invention, the second information includes: first identification information; determining the corresponding third terminal based on the second information includes: determining the corresponding third terminal based on the first identification information.

[0315] The implementation embodiments of the above-mentioned server-side information processing method are all applicable to the embodiments of the information processing device and can achieve the same technical effect.

[0316] This invention also provides an information processing device applied to cloud devices, such as... Figure 19 As shown, it includes:

[0317] The third receiving module 191 is used to receive third information sent by the server in response to a sharing request; the sharing request is a sharing request for at least a first portion of the screen of the first terminal; the third information includes: terminal identification information of the second terminal;

[0318] The first connection establishment module 192 is used to establish a connection with the second terminal based on the terminal identification information of the second terminal;

[0319] The sixth sending module 193 is used to send data for the at least first part of the screen image to the second terminal through the connection.

[0320] The information processing device provided in this embodiment of the invention receives third information from a server regarding a sharing request; the sharing request is a sharing request for at least a first portion of the screen image of a first terminal; the third information includes: terminal identification information of a second terminal; a connection is established with the second terminal based on the terminal identification information of the second terminal; data regarding the at least first portion of the screen image is sent to the second terminal through the connection; it can support the transformation of multiple people operating one terminal to collaboratively play games, for example, it can enable multiple people in a family to collaboratively play games through the same TV, and transfer the controllers of some people to computers, mobile phones, or other terminals in other rooms, so that they can continue to play games collaboratively in different rooms; thereby meeting the diverse needs of users, facilitating user use, making full use of the terminals around them, and effectively solving the problem that existing information processing solutions for multiplayer cloud games cannot meet the diverse needs of users.

[0321] The third information further includes: first response information; the first response information includes: first response information of the second terminal agreeing to the sharing request; the step of establishing a connection with the second terminal based on the terminal identification information of the second terminal includes: establishing a connection with the second terminal based on the terminal identification information of the second terminal and the first response information.

[0322] Furthermore, the first response information also includes at least one of the resolution and bandwidth corresponding to the second terminal; sending data for the at least first part of the screen image to the second terminal through the connection includes: determining the maximum bit rate of data transmission corresponding to the second terminal based on the resolution and bandwidth; and sending data for the at least first part of the screen image to the second terminal through the connection based on the resolution, bandwidth, and maximum bit rate.

[0323] In this embodiment of the invention, sending data for the at least first portion of the screen display to the second terminal via the connection includes: receiving an operation instruction for the at least first portion of the screen display sent by the second terminal via the connection; and sending data for the at least first portion of the screen display to the second terminal via the connection according to the operation instruction sent by the second terminal.

[0324] Furthermore, the information processing device further includes: a fourth receiving module, configured to receive fourth information sent by the server in response to the merging request; the merging request is a merging request for at least a second portion of the screen image of the first terminal; the fourth information includes: terminal identification information of the third terminal; and a second processing module, configured to disconnect the connection with the third terminal based on the terminal identification information of the third terminal.

[0325] The fourth information further includes: second response information; the second response information includes: a second response information in which the third terminal agrees to the merging request; the step of disconnecting the connection with the third terminal based on the terminal identification information of the third terminal includes: disconnecting the connection with the third terminal based on the terminal identification information of the third terminal and the second response information.

[0326] Furthermore, the information processing device further includes: a first determining module, configured to determine first identification information based on the terminal identification information of the third terminal after disconnecting the connection with the third terminal based on the terminal identification information of the third terminal; and a second determining module, configured to determine that a first control device is connected to the first terminal based on the first identification information and a first mapping relationship; wherein the first mapping relationship is a mapping relationship between the first identification information and the first control device that controls the at least second part of the screen display.

[0327] In this embodiment of the invention, the information processing device further includes: a third processing module, configured to process an operation instruction corresponding to the second information sent by the first terminal after determining that the first control device is connected to the first terminal based on the first identification information and the first mapping relationship; wherein the second information includes the first identification information; the second information is obtained by the first terminal based on the merging request.

[0328] The aforementioned implementation embodiments of the information processing method on the cloud device side are all applicable to the embodiments of the information processing device, and can achieve the same technical effect.

[0329] This invention also provides an information processing device, including a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, it implements the aforementioned information processing method on the first terminal side; or, when the processor executes the program, it implements the aforementioned information processing method on the server side; or, when the processor executes the program, it implements the aforementioned information processing method on the cloud device side.

[0330] The implementation embodiments of the information processing methods on the first terminal side, server side, or cloud device side described above are all applicable to the embodiments of the information processing device and can achieve the same technical effect.

[0331] This invention also provides a readable storage medium storing a program that, when executed by a processor, implements the steps in the first terminal-side information processing method described above; or, when executed by a processor, implements the steps in the server-side information processing method described above; or, when executed by a processor, implements the steps in the cloud device-side information processing method described above.

[0332] The implementation embodiments of the information processing methods on the first terminal side, server side, or cloud device side described above are all applicable to the embodiments of the readable storage medium and can achieve the same technical effect.

[0333] It should be noted that many of the functional components described in this specification are referred to as modules in order to more specifically emphasize the independence of their implementation.

[0334] In this embodiment of the invention, the module can be implemented in software so that it can be executed by various types of processors. For example, an identified executable code module may include one or more physical or logical blocks of computer instructions, which may be constructed as objects, procedures, or functions. Nevertheless, the executable code of the identified module does not need to be physically located together, but may include different instructions stored in different bits, which, when logically combined, constitute the module and achieve the module's intended purpose.

[0335] In practice, an executable code module can be a single instruction or many instructions, and can even be distributed across multiple different code segments, different programs, and across multiple memory devices. Similarly, operational data can be identified within the module and can be implemented in any suitable form and organized within any suitable type of data structure. This operational data can be collected as a single dataset or distributed across different locations (including different storage devices), and can exist, at least in part, solely as electronic signals within the system or network.

[0336] When a module can be implemented using software, considering the current level of hardware technology, modules that can be implemented in software can be implemented using hardware circuits by those skilled in the art to achieve the corresponding functions, without considering cost. These hardware circuits include conventional very-large-scale integrated circuits (VLSI) or gate arrays, as well as existing semiconductors such as logic chips and transistors, or other discrete components. Modules can also be implemented using programmable hardware devices, such as field-programmable gate arrays, programmable array logic, and programmable logic devices.

[0337] The above describes the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An information processing method, applied to a first terminal, characterized in that, include: Based on the sharing request, obtain at least one piece of initial information; The first information includes at least one of: second identification information of the first user and a second mapping relationship; wherein, the second mapping relationship is a mapping relationship between the second identification information and a second control device that controls at least a first portion of the screen of the first terminal; Send the sharing request and the first information to the server; The sharing request is a sharing request for at least a first part of the screen of the first terminal, where the at least first part of the screen is the entire screen or a portion of the screen.

2. The information processing method according to claim 1, characterized in that, After sending the sharing request and the first information to the server, the process also includes: Stop sending the operation instructions corresponding to the first information received to the cloud device.

3. The information processing method according to claim 1, characterized in that, Also includes: Based on the merge request, obtain at least one piece of second information; Send the merge request and the second information to the server; The merging request is a merging request for at least a second portion of the screen display of the first terminal.

4. The information processing method according to claim 3, characterized in that, After sending the merge request and the second information to the server, the process further includes: Send the operation command corresponding to the received second information to the cloud device.

5. An information processing method applied to a server, characterized in that, include: Receive a sharing request and at least one first piece of information sent by the first terminal; The first information includes at least one of: second identification information of the first user and a second mapping relationship; wherein, the second mapping relationship is a mapping relationship between the second identification information and a second control device that controls at least a first portion of the screen of the first terminal; Based on the first information, send third information regarding the sharing request to the cloud device; The sharing request is a sharing request for at least a first part of the screen of the first terminal, where the at least first part of the screen is the entire screen or a portion of the screen. The third information includes: the terminal identification information of the second terminal; The second terminal is the terminal associated with the first information.

6. The information processing method according to claim 5, characterized in that, The step of sending third information to the cloud device in response to the sharing request based on the first information includes: Based on the first information, the corresponding second terminal is determined; Send the sharing request to the second terminal; The system receives first response information from the second terminal, the first response information including: a first response information in which the second terminal agrees to the sharing request; The first response information and the terminal identification information of the second terminal are sent to the cloud device as the third information.

7. The information processing method according to claim 6, characterized in that, The first response information also includes at least one of the resolution and bandwidth corresponding to the second terminal.

8. The information processing method according to claim 6, characterized in that, Also includes: Receive a merging request and at least one second piece of information sent by the first terminal; Based on the second information, send the fourth information regarding the merging request to the cloud device; The merging request is a merging request for at least a second portion of the screen image of the first terminal. The fourth piece of information includes: the terminal identification information of the third terminal; The third terminal is the terminal associated with the second information.

9. The information processing method according to claim 8, characterized in that, The step of sending fourth information to the cloud device regarding the merging request based on the second information includes: Based on the second information, the corresponding third terminal is determined; The merge request is sent to the third terminal; Receive a second response from the third terminal indicating agreement to the merger request; The second response information and the terminal identification information of the third terminal are sent to the cloud device as the fourth information.

10. An information processing method applied to a cloud device, characterized in that, include: Receive third-party information from the server in response to the sharing request; The sharing request is a sharing request for at least a first portion of the screen of the first terminal; The third information includes: terminal identification information of the second terminal; the third information is sent according to the first information, the first information including: at least one of the second identification information of the first user and the second mapping relationship; wherein, the second mapping relationship is the mapping relationship between the second identification information and the second control device that controls at least a first part of the screen of the first terminal, and the at least a first part of the screen is the entire screen or a part of the screen. A connection is established with the second terminal based on the terminal identification information of the second terminal; Through the connection, data concerning at least the first portion of the screen image is sent to the second terminal.

11. The information processing method according to claim 10, characterized in that, The third information also includes: first response information, which includes: a first response information in which the second terminal agrees to the sharing request; The step of establishing a connection with the second terminal based on the terminal identification information of the second terminal includes: A connection is established with the second terminal based on the terminal identification information and the first response information.

12. The information processing method according to claim 11, characterized in that, The first response information also includes at least one of the resolution and bandwidth corresponding to the second terminal; Sending data for at least the first portion of the screen display to the second terminal via the connection includes: Based on the resolution and bandwidth, determine the maximum data transmission rate corresponding to the second terminal; Based on the resolution, bandwidth, and maximum bitrate, data for at least the first portion of the screen image is sent to the second terminal via the connection.

13. The information processing method according to claim 10, characterized in that, Sending data for at least the first portion of the screen display to the second terminal via the connection includes: Through the connection, the system receives operation instructions sent by the second terminal for at least the first portion of the screen display. According to the operation instructions sent by the second terminal, data for the at least first part of the screen image is sent to the second terminal through the connection.

14. The information processing method according to claim 10, characterized in that, Also includes: Receive the fourth information sent by the server in response to the merge request; The merging request is a merging request for at least a second portion of the screen image of the first terminal. The fourth piece of information includes: the terminal identification information of the third terminal; The connection with the third terminal is disconnected based on the terminal identification information of the third terminal.

15. The information processing method according to claim 14, characterized in that, After disconnecting the connection with the third terminal based on the terminal identification information of the third terminal, the process further includes: Based on the terminal identification information of the third terminal, the first identification information is determined; Based on the first identification information and the first mapping relationship, it is determined that the first control device is connected to the first terminal; Wherein, the first mapping relationship is the mapping relationship between the first identification information and the first control device that controls the at least second part of the screen.

16. The information processing method according to claim 15, characterized in that, After determining that the first control device is connected to the first terminal based on the first identification information and the first mapping relationship, the method further includes: Process the operation instructions corresponding to the second information sent by the first terminal; The second information includes the first identification information; The second information is obtained by the first terminal based on the merging request.

17. An information processing device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that, When the processor executes the program, it implements the information processing method as described in any one of claims 1 to 4; or... When the processor executes the program, it implements the information processing method as described in any one of claims 5 to 9; or... When the processor executes the program, it implements the information processing method as described in any one of claims 10 to 16.

18. A readable storage medium having a program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the information processing method as described in any one of claims 1 to 4; or... When the program is executed by the processor, it implements the steps of the information processing method as described in any one of claims 5 to 9; or, When the program is executed by the processor, it implements the steps of the information processing method as described in any one of claims 10 to 16.

Images