A display method, a terminal, and a system

Abstract

The present application discloses a display method, a terminal and a system. When a first operation on a client on the terminal is received, the terminal sends a request for requesting a matching resource to the server. The server matches a first resource for the terminal according to the request and sends the identification information of the first resource to the terminal. When a second operation on the client on the terminal is received, the terminal sends a message to the server, and the message includes an operation instruction corresponding to the second operation and the identification information of the first resource. The server renders the video and audio streams and sends all the video and audio streams corresponding to the identification information of the first resource to the terminal, and the terminal displays an interface formed by all the video and audio streams corresponding to the identification information of the first resource. In this way, the server configures resources for each terminal, and sends the video and audio streams of the same resource to all the clients corresponding to the same resource. Each client can play all the video and audio of the same resource, realizing a high-performance immersive gaming experience and reducing latency.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of communication technologies, and in particular, to a display method, a terminal, and a system. Background Art

[0002] With the development of cloud computing and the full application of the fifth-generation mobile communication technology (5G) network, the development of cloud services has been promoted. Among them, cloud services can include cloud gaming, cloud monitoring, cloud security, and so on. Taking cloud gaming as an example, cloud gaming can include single-player games and battle games. In the battle game scenario, both viewers and game players hope to see the battle live. Therefore, how to display the battle live of a battle game on one screen is a technical problem to be solved by those skilled in the art. Summary of the Invention

[0003] A display method, a terminal, and a system provided by the embodiments of the present application can display the battle live of a battle game on one screen.

[0004] To achieve the above object, the embodiments of the present application adopt the following technical solutions.

[0005] In a first aspect, a display method provided by the embodiments of the present application is applied to a terminal. The execution subject of the method can be the terminal or a component located in the terminal (such as a chip, a chip system, or a processor, etc.). Hereinafter, the execution subject is the first terminal as an example for description. The method includes: when receiving a first operation on a client on the first terminal, the first terminal sends a first request to a cloud server, and the first request is used to request matching resources. The first terminal receives the identification information of a first resource returned by the cloud server, and the first resource is matched by the cloud server for the first terminal according to the first request. When receiving a second operation on the client on the first terminal, the first terminal sends a first message to the cloud server, and the first message includes an operation instruction corresponding to the second operation, information of the first terminal, and the identification information of the first resource. The first terminal receives all video and audio streams corresponding to the identification information of the first resource returned by the cloud server, and the video and audio streams are rendered by the cloud server. The first terminal displays a first interface, and the first interface is formed by all the video and audio streams corresponding to the identification information of the first resource.

[0006] In the embodiments of the present application, resources are configured for each terminal by the first server, so that when the second server pushes video and audio streams, the pushing is based on the resources configured by the first server. In this way, the second server can push all the video and audio streams corresponding to the same resource to all the clients corresponding to the resource, so that each client can see the interfaces of other clients, realizing a high-performance immersive gaming experience with low latency. In addition, there is no need to merge and process the video and audio streams of each terminal on the cloud server, reducing the load on the cloud server. In addition, game developers only need to embed the client corresponding to the first server, such as a battle client, on the client, and deploy the game application on the cloud server, so that single-player games can be changed into battle games at low cost.

[0007] In a specific implementable manner, the first terminal has a first client and a second client. When a first operation on the first client on the first terminal is received, the first terminal sends a first request to the cloud server; when a second operation on the second client on the first terminal is received, the first terminal sends a first message to the cloud server.

[0008] In some implementable manners, the communication system may include the above-mentioned first terminal and the cloud server, and the communication system further includes a second terminal, and the second terminal has a first client. The first request is further used to request the cloud server to match a first resource for the second terminal, and the information of the second terminal is carried in the first request.

[0009] In some implementable manners, the information of the first terminal includes the connection address of the first terminal. Based on the connection address of the first terminal, a communication connection is established between the first terminal and the cloud server.

[0010] In a specific implementable manner, the first terminal displays a first interface. Specifically, the first terminal displays the first interface according to a preset display rule.

[0011] In a specific implementable manner, the preset display rule includes: a split-screen display rule, and the split-screen display rule is a rule for dividing the display area of the display screen according to a preset ratio; it can be seen that the terminal uses the split-screen display rule to display the interfaces formed by the video and audio streams of each terminal, which is more suitable for large-screen terminals, such as Huawei Smart Screens, etc., bringing a low-latency and high-performance immersive gaming experience to game players and audiences through the split-screen display method.

[0012] In a specific implementable manner, the preset display rule includes: a screen switching display rule, which is a rule for switching and displaying the content shown in the display area of the display screen after the terminal receives a screen switching operation. It can be seen that the terminal uses the screen switching display rule to display the interface formed by the video and audio streams of each terminal, which is more suitable for terminals with a relatively small display area, such as mobile phones and tablets, thereby enhancing the screen switching cloud game battle experience on terminals with a relatively small display area such as mobile phones and tablets.

[0013] In a second aspect, a display method provided by an embodiment of the present application is applied to a communication system. The communication system includes a first terminal and a cloud server. The method includes: when a first operation on a client on the first terminal is received, the first terminal sends a first request to the cloud server. The first request is used to request a matching resource. The cloud server receives the first request. The cloud server matches a first resource for the first terminal according to the first request. The cloud server returns the identification information of the first resource to the first terminal. The first terminal receives the identification information of the first resource. When a second operation on the client on the first terminal is received, the first terminal sends a first message to the cloud server. The first message includes an operation instruction corresponding to the second operation, information of the first terminal, and the identification information of the first resource. The cloud server renders the video and audio streams according to the first message. The cloud server sends all the video and audio streams corresponding to the identification information of the first resource to the first terminal. The first terminal receives all the video and audio streams corresponding to the identification information of the first resource. The first terminal displays a first interface, and the first interface is formed by all the video and audio streams corresponding to the identification information of the first resource.

[0014] In the embodiment of the present application, by configuring resources for each terminal through the first server, when the second server pushes video and audio streams, it pushes based on the resources configured by the first server. In this way, the second server can push all the video and audio streams corresponding to the same resource to all the clients corresponding to this resource, so that each client can see the interfaces of other clients, achieving a high-performance immersive game experience with low latency. In addition, it is not necessary to merge and process the video and audio streams of each terminal on the cloud server, reducing the load on the cloud server. In addition, game developers only need to embed the client corresponding to the first server, such as a battle client, on the client, and deploy the game application program on the cloud server, so that a single-player game can be changed into a battle game at low cost.

[0015] In a specific implementable manner, the cloud server includes a first server and a second server. The first server receives a first request sent by a first terminal, matches a first resource for the first terminal according to the first request, and returns the identification information of the first resource to the first terminal; the second server receives a first message sent by the first terminal, renders a video and audio stream according to the first message, and sends all the video and audio streams corresponding to the identification information of the first resource to the first terminal.

[0016] Among them, the first server and the second server can be the same server or different servers. No specific limitation is made here.

[0017] In some implementable manners, it further includes: the first server sends a second request to the second server, and the second request is used to request to allocate an application program for the first terminal, and the identification information of the first resource and the information of the first terminal are carried in the second request; the second server allocates an application program for the first terminal according to the second request.

[0018] Among them, the application program can be understood as an application deployed on the cloud server. Exemplarily, the second client on the terminal can be understood as a game center, and the application program can be understood as an application of a certain game in the game center. It can be displayed in the form of an application icon on the second client.

[0019] In some implementable manners, it further includes: the second server establishes an association relationship among the identification information of the first resource, the information of the first terminal, and the application program. In this way, on the second server, an association relationship among the identification information of the first resource, the information of the terminal, and the application program is established. When the second server receives a message reported by the first terminal, it can find the first resource and the application program corresponding to the first terminal according to the association relationship.

[0020] In a specific implementable manner, the first terminal has a first client and a second client, the first client corresponds to the first server, and the second client corresponds to the second server. When a first operation on the first client on the first terminal is received, the first terminal sends a first request to the first server; when a second operation on the second client on the first terminal is received, the first terminal sends a first message to the second server.

[0021] In some implementable manners, the communication system further includes a second terminal; the second terminal has a first client and a second client. When a first operation on the first client on the first terminal is received, the first terminal sends a third request to the first server, and the third request is used to request to invite the first client of the second terminal. The first server matches a first resource for the second terminal according to the third request.

[0022] In some implementable manners, the information of the first terminal includes the connection address of the first terminal; further included is: based on the connection address of the first terminal, a communication connection is established between the first terminal and the cloud server.

[0023] In a specific implementable manner, the first terminal displays a first interface, specifically: the first terminal displays the first interface according to a preset display rule.

[0024] In a specific implementable manner, the preset display rule includes: a split-screen display rule, and the split-screen display rule is a rule for dividing the display area of the display screen according to a preset ratio; it can be seen that the terminal uses the split-screen display rule to display the interface formed by the video and audio streams of each terminal, which is more suitable for large-screen terminals, such as Huawei Smart Screens, etc., and brings a low-latency and high-performance immersive gaming experience to game players and audiences through the split-screen display method.

[0025] In a specific implementable manner, the preset display rule includes: a screen-switching display rule, and the screen-switching display rule is a rule for switching the content displayed in the display area of the display screen after the terminal receives a screen-switching operation. It can be seen that the terminal uses the screen-switching display rule to display the interface formed by the video and audio streams of each terminal, which is more suitable for terminals with a smaller display area, such as mobile phones and tablets, etc., thereby increasing the screen-switching cloud gaming battle experience on terminals with a smaller display area such as mobile phones and tablets.

[0026] In a third aspect, a terminal provided in an embodiment of the present application may include: a sending module, a receiving module, and a display module. Among them,

[0027] The sending module is used to send a first request to the cloud server when receiving a first operation on the client on the first terminal, and the first request is used to request matching resources;

[0028] The receiving module is used to receive the identification information of the first resource returned by the cloud server, and the first resource is matched by the cloud server for the first terminal according to the first request;

[0029] The sending module is used to send a first message to the cloud server when receiving a second operation on the client on the first terminal, and the first message includes an operation instruction corresponding to the second operation, the information of the first terminal, and the identification information of the first resource;

[0030] The receiving module is used to receive all the video and audio streams corresponding to the identification information of the first resource, and the video and audio streams are rendered by the cloud server;

[0031] The display module is used to display a first interface, and the first interface is formed by all the video and audio streams corresponding to the identification information of the first resource.

[0032] In the embodiments of the present application, the first server configures resources for each terminal, so that when the second server pushes video and audio streams, the pushing is based on the resources configured by the first server. In this way, the second server can push all the video and audio streams corresponding to the same resource to all the clients corresponding to the resource, so that each client can see the interfaces of other clients, realizing a high-performance immersive gaming experience with low latency. In addition, it is not necessary to merge and process the video and audio streams of each terminal on the cloud server, reducing the load on the cloud server. In addition, game developers only need to embed the client corresponding to the first server, such as a battle client, on the client side, and deploy the game application on the cloud server, so that single-player games can be changed into battle games at low cost.

[0033] In a specific implementable manner, the first terminal has a first client and a second client.

[0034] The sending module is configured to send a first request to the cloud server when a first operation on the first client on the first terminal is received; and send a first message to the cloud server when a second operation on the second client on the first terminal is received.

[0035] In some implementable manners, the communication system further includes a second terminal, and the second terminal has a first client; the first request is further used to request the cloud server to match a first resource for the second terminal, and the information of the second terminal is carried in the first request.

[0036] In some implementable manners, the information of the first terminal includes the connection address of the first terminal; further included is a establishing module configured to establish a communication connection between the first terminal and the cloud server based on the connection address of the first terminal.

[0037] In a specific implementable manner, the display module is configured to display a first interface according to a preset display rule.

[0038] In a specific implementable manner, the preset display rule includes: a split-screen display rule, and the split-screen display rule is a rule for dividing the display area of the display screen according to a preset ratio; it can be seen that the terminal adopts the split-screen display rule to display the interfaces formed by the video and audio streams of each terminal, which is more suitable for large-screen terminals, such as Huawei Smart Screens, etc., bringing a low-latency and high-performance immersive gaming experience to game players and viewers through the split-screen display method.

[0039] In a specific implementable manner, the preset display rule includes: a screen switching display rule, which is a rule for switching and displaying the content displayed in the display area of the display screen after the terminal receives a screen switching operation. It can be seen that the terminal uses the screen switching display rule to display the interface formed by the video and audio streams of each terminal, which is more suitable for terminals with a smaller display area, such as mobile phones and tablets, thereby enhancing the screen switching cloud game battle experience on terminals with a smaller display area such as mobile phones and tablets.

[0040] Fourthly, a system provided by an embodiment of the present application includes: a first terminal and a cloud server;

[0041] The first terminal is configured to execute: when receiving a first operation on the client on the first terminal, send a first request to the cloud server, and the first request is used to request a matching resource.

[0042] The cloud server is configured to execute: receive the first request; match a first resource for the first terminal according to the first request; return the identification information of the first resource to the first terminal.

[0043] The first terminal is further configured to execute: receive the identification information of the first resource; when receiving a second operation on the client on the first terminal, send a first message to the cloud server, and the first message includes an operation instruction corresponding to the second operation, information of the first terminal, and the identification information of the first resource.

[0044] The cloud server is further configured to execute: render video and audio streams according to the first message; send all video and audio streams corresponding to the identification information of the first resource to the first terminal.

[0045] The first terminal is further configured to execute: receive all video and audio streams corresponding to the identification information of the first resource; display a first interface, and the first interface is formed by all video and audio streams corresponding to the identification information of the first resource.

[0046] In the embodiment of the present application, resources are configured for each terminal by the first server, so that when the second server pushes video and audio streams, the pushing is based on the resources configured by the first server. In this way, the second server can push all video and audio streams corresponding to the same resource to all clients corresponding to the resource, so that each client can see the interfaces of other clients, realizing a high-performance immersive game experience with low latency. In addition, it is not necessary to merge and process the video and audio streams of each terminal on the cloud server, reducing the load on the cloud server. In addition, game developers only need to embed the client corresponding to the first server, such as a battle client, on the client, and deploy the game application on the cloud server, so that single-player games can be changed into battle games at low cost.

[0047] In a specific implementable manner, the cloud server includes a first server and a second server;

[0048] The first server is used to execute: receiving a first request sent by a first terminal, matching a first resource for the first terminal according to the first request, and returning the identification information of the first resource to the first terminal

[0049] The second server is used to execute: receiving a first message sent by the first terminal, rendering a video and audio stream according to the first message, and sending all the video and audio streams corresponding to the identification information of the first resource to the first terminal.

[0050] In a specific implementable manner, the first server is further used to execute: sending a second request to the second server, where the second request is used to request to allocate an application for the first terminal, and the identification information of the first resource and the information of the first terminal are carried in the second request.

[0051] The second server is further used to execute: allocating an application for the first terminal according to the second request.

[0052] In a specific implementable manner, the second server is further used to execute: establishing an association relationship among the identification information of the first resource, the information of the first terminal, and the application.

[0053] In a specific implementable manner, the first terminal has a first client and a second client, the first client corresponds to the first server, and the second client corresponds to the second server; the first terminal is further used to execute:

[0054] When receiving a first operation on the first client on the first terminal, sending a first request to the first server;

[0055] When receiving a second operation on the second client on the first terminal, sending a first message to the second server.

[0056] In some implementable manners, the communication system further includes a second terminal; the second terminal has a first client and a second client; the first terminal is further used to execute: when receiving a first operation on the first client on the first terminal, sending a third request to the first server, where the third request is used to request to invite the first client of the second terminal;

[0057] The first server is further used to execute: matching a first resource for the second terminal according to the third request.

[0058] In a specific implementable manner, the information of the first terminal includes the connection address of the first terminal; the first terminal is further used to execute: based on the connection address of the first terminal, establishing a communication connection between the first terminal and the cloud server.

[0059] In a specific implementable manner, the first terminal is further configured to: display a first interface according to a preset display rule.

[0060] In a specific implementable manner, the preset display rule includes: a split-screen display rule, which is a rule for dividing the display area of the display screen according to a preset ratio; it can be seen that the terminal uses the split-screen display rule to display the interface formed by the video and audio streams of each terminal, which is more suitable for large-screen terminals, such as Huawei Smart Screens, etc., and brings a low-latency and high-performance immersive gaming experience to game players and audiences through the split-screen display method.

[0061] In a specific implementable manner, the preset display rule includes: a screen-switching display rule, which is a rule for switching the content displayed in the display area of the display screen after the terminal receives a screen-switching operation. It can be seen that the terminal uses the screen-switching display rule to display the interface formed by the video and audio streams of each terminal, which is more suitable for terminals with a smaller display area, such as mobile phones and tablets, etc., thereby enhancing the screen-switching cloud gaming battle experience on terminals with a smaller display area such as mobile phones and tablets.

[0062] In a fifth aspect, a terminal provided by an embodiment of the present application includes a touch screen, a memory, a transceiver, one or more processors, multiple application programs, and one or more programs; wherein, the one or more programs are stored in the memory, and the transceiver is used for transmitting or receiving wireless signals; when the one or more processors execute the one or more programs, the terminal implements the method as described in the first aspect.

[0063] In a sixth aspect, a computer-readable storage medium provided by an embodiment of the present application includes computer instructions, and when the computer instructions run on a terminal, the terminal is caused to execute the method as described in the first aspect, or the method as described in the second aspect.

[0064] In a seventh aspect, a chip system provided by an embodiment of the present application includes one or more processors, and when the one or more processors execute instructions, the one or more processors execute the method as described in the first aspect, or the method as described in the second aspect.

[0065] In an eighth aspect, a computer program product provided by an embodiment of the present application, when the computer program product runs on a computer, causes the computer to execute the method as described in the first aspect, or the method as described in the second aspect.

[0066] Among them, for the specific implementation manners and corresponding technical effects of each embodiment in the second to eighth aspects above, reference can be made to the specific implementation manners and technical effects of the first aspect above.

[0067] In an embodiment of the present application, the first server configures resources for each terminal, so that when the second server pushes video and audio streams, the pushing is based on the resources configured by the first server. In this way, the second server can push all the video and audio streams corresponding to the same resource to all the clients corresponding to the resource, so that each client can see the interfaces of other clients, achieving a high-performance immersive gaming experience with low latency. In addition, it is not necessary to merge and process the video and audio streams of each terminal on the cloud server, reducing the load on the cloud server. In addition, game developers only need to embed the client corresponding to the first server, such as a battle client, on the client, and deploy the game application on the cloud server, so that a single-player game can be changed into a battle game at low cost. Brief Description of the Drawings

[0068] Figure 1 It is a schematic structural diagram of a communication system;

[0069] Figure 2 It is a schematic flowchart of a display method;

[0070] Figure 3 It is a schematic flowchart of another display method;

[0071] Figure 4 It is a schematic flowchart of another display method;

[0072] Figure 5A It is a schematic structural diagram of a communication system provided by an embodiment of the present application;

[0073] Figure 5B It is a schematic flowchart of a display method provided by an embodiment of the present application;

[0074] Figure 6 It is a schematic structural diagram of a terminal provided by an embodiment of the present application;

[0075] Figure 7A It is a schematic flowchart of a display method provided by an embodiment of the present application;

[0076] Figure 7B It is a schematic flowchart of a display method provided by an embodiment of the present application;

[0077] Figure 8 It is a schematic diagram of an interface of a terminal provided by an embodiment of the present application;

[0078] Figure 9 It is a schematic diagram of another interface of a terminal provided by an embodiment of the present application;

[0079] Figure 10 It is a schematic diagram of another interface of a terminal provided by an embodiment of the present application;

[0080] Figure 11 A schematic diagram of another interface of a terminal provided by an embodiment of the present application;

[0081] Figure 12 A schematic diagram of another interface of a terminal provided by an embodiment of the present application;

[0082] Figure 13 A schematic diagram of another interface of a terminal provided by an embodiment of the present application;

[0083] Figure 14 A schematic diagram of another interface of a terminal provided by an embodiment of the present application;

[0084] Figure 15 A schematic diagram of another interface of a terminal provided by an embodiment of the present application;

[0085] Figure 16 A schematic diagram of another interface of a terminal provided by an embodiment of the present application;

[0086] Figure 17 A schematic diagram of another interface of a terminal provided by an embodiment of the present application;

[0087] Figure 18 A schematic diagram of another interface of a terminal provided by an embodiment of the present application;

[0088] Figure 19 A schematic diagram of another interface of a terminal provided by an embodiment of the present application;

[0089] Figure 20 A schematic diagram of another interface of a terminal provided by an embodiment of the present application;

[0090] Figure 21 A schematic diagram of the structure of another terminal provided by an embodiment of the present application. Detailed implementation manners

[0091] Figure 1 A communication system composed of a terminal and a cloud server. As Figure 1 shown, the communication system 100 may include a cloud server 110 (such as server 11, server 12, server 13, and server 14...) and at least one terminal 120 (such as terminal 21, terminal 22, terminal 23, terminal 24...).

[0092] The cloud server 110 may include a cloudified business resource server, a cloud virtual device, an application APP, and a cloudified server. Among them, the cloudified business resource server is a physical server in cloud computing, responsible for providing computing resources directly used by the business, that is, the cloud virtual device. The cloud virtual device is a virtual device and system created in the cloud server based on cloud computing-related technologies for running terminal application software. For example, a cloud phone running the Android system. The APP is terminal application software, such as: game client, office software client, and so on. The cloudified server is a set of server-side software deployed by the cloudified business operator on the cloud virtual device. Its main functions include: controlling the start and stop of terminal application software on the cloud virtual device; implementing audio and video capture, encoding, and forwarding; restoring the user operations reported by the cloudified business client on the cloud virtual device to control the terminal application software, such as: cloud game services in the cloud game scenario.

[0093] The terminal 120 is used to install and run various business application software. Exemplarily, the terminal 120 includes a cloudified business client. The cloudified business client is a specific terminal application software provided by the cloudified business operator to users. Through this application, users can access one or more terminal application software deployed by the cloudified business operator on the cloud virtual device, such as: cloud game client, cloud office client, and so on.

[0094] Among them, the above terminal 120 may be a mobile phone, a tablet computer, a desktop type, a laptop, a notebook computer, an ultra-mobile personal computer (UMPC), a handheld computer, a smart TV, a netbook, a virtual reality device, and other devices with a display function. In the embodiments of the present application, no special restrictions are imposed on the specific form of the terminal.

[0095] Based on Figure 1 the structural schematic diagram of the shown communication system, the current display method will be introduced.

[0096] With the development of cloud computing and the full application of 5G networks, the development of cloud services has been promoted. Among them, cloud services can include cloud games, cloud monitoring, cloud security, and so on. Taking cloud games as an example, cloud games can be further divided into cloud mobile games, cloud console games, and cloud personal computer (PC) games. Among them, cloud mobile games can include mobile phone games, tablet games, smart TV games, etc., which usually run on the advanced risc machine (ARM) architecture. Cloud console games (such as console games) and cloud PC games (such as PC games) usually run on the x86 (the X86 architecture) architecture. Whether it is cloud mobile games, cloud console games, or cloud PC games, they can all include single-player games and battle games. Among them, single-player games refer to single-player games, and battle games refer to multiplayer games.

[0097] In the battle game scenario, both spectators and game players hope to see the live battle situation. To this end, the following methods can be used to achieve the display of the live battle situation of battle games:

[0098] The first method, Figure 2 is a flowchart of a display method. As Figure 2 shown, the game application (APP) runs on the cloud server (i.e., the cloud game host, or cloud phone). ① When the client on the terminal (such as a mobile phone, tablet computer, notebook computer, desktop computer, etc.) receives the player's operation and sends an operation instruction to the cloud server in response to this operation. ② The cloud server executes the player's operation according to the operation instruction and renders the game scene into a video and audio stream. Then, the cloud server pushes the video and audio stream to the client. ③ The client plays the video and audio stream. That is to say, when multiple players play a game battle, the client of each player obtains the player's operation and sends the operation instruction corresponding to this operation to each cloud server (i.e., the cloud server cluster). The cloud server cluster executes the operation instructions uploaded by the clients of each player and renders them into a video and audio stream, and pushes the video and audio stream to the clients of each player respectively, and the clients of each player play the video and audio stream of each player. In this way, the battle screen of each player is displayed on the player's device.

[0099] The second method, Figure 3 is a flowchart of a display method. As Figure 3As shown in the figure, ① is the same as ① in the above first method. The difference compared with the first method is that ② after each cloud server renders the video and audio streams of each player, the encoding server of each cloud server encodes the video and audio streams of the player and pushes the encoded video and audio streams to the push server. The push server merges the video and audio streams of each player to obtain the merged video and audio stream and sends the merged video and audio stream to all clients. ③ These clients all play the merged video and audio stream. That is to say, when multiple players are in a game battle, the client of each player obtains the player's operation and sends the operation instruction corresponding to the operation to each cloud server (i.e., the cloud server cluster). The cloud server cluster executes the operation instructions uploaded by the clients of each player and renders them into video and audio streams. The encoding server in the cloud server cluster merges the video and audio streams of each player to obtain the merged video and audio stream and pushes the merged video and audio stream to each client through the push server, and the clients of each player play the merged video and audio stream. In this way, the battle scenes of all players are displayed on the player's device.

[0100] The third method Figure 4 is a schematic flowchart of a display method, as Figure 4 shown. The difference compared with the first method and the second method is that the game application runs on the terminal. ① When the client on the terminal receives the player's operation and sends an operation instruction to the cloud server in response to the operation. ② The cloud server pushes the operation instructions of all players to the client of the player. ③ The terminal executes the operation instructions of each player and renders them into the video and audio streams of each player. After that, the client on the terminal plays the video and audio streams of each player. That is to say, when multiple players are in a game battle, the client of each player obtains the player's operation and sends the operation instruction corresponding to the operation to each cloud server (i.e., the cloud server cluster). The cloud server cluster pushes the operation instructions of all players to the client of each player. The client of each player executes the operation instruction and renders the video and audio streams of each player. After that, the client of each player plays the video and audio streams of each player. In this way, the battle scenes of all players are displayed on the player's device.

[0101] In summary, in the first method, players cannot see the live battle situation of other players, resulting in a poor immersive gaming experience. In the second method, players can see the live battle situation of other players, but the cloud server cluster needs to merge the video and audio streams of each player, increasing the load on the cloud server and having a large latency. In the third method, players can see the live battle situation of other players, but each player's terminal needs to execute operation instructions and render the video and audio streams of each player, with poor execution efficiency and a large latency. It can be seen that currently, there are defects such as a large latency and a low-performance immersive gaming experience in realizing the display of the live battle situation of battle games.

[0102] To solve the above technical problems, the present application provides a display method, which is applied to a communication system. The communication system includes a first terminal, a first server, and a second server. The method includes: when a first operation on a first client of the first terminal is received, the first terminal sends a first request to the first server. The first request is used to request matching resources. The first server matches a first resource for the first terminal according to the request and returns the identification information of the first resource to the first terminal. The first server sends a second request to the second server. The second request is used to request the allocation of an application program, and the identification information of the first resource and the information of the first terminal are carried in the second request. When a second operation on the first client of the first terminal is received, the first terminal sends a first message to the second server. The message includes the operation instruction corresponding to the second operation, the information of the first terminal, and the identification information of the first resource. The second server executes the operation corresponding to the operation instruction and renders the video and audio stream. The second server sends all the video and audio streams corresponding to the identification information of the first resource to the first terminal. The first client of the first terminal plays the video and audio and displays an interface, which is formed by all the video and audio streams corresponding to the identification information of the first resource. In this way, by the first server configuring resources for each terminal, based on the configured resources, the cloud server can send the video and audio streams of the same resource to all the clients corresponding to the same resource, and all the clients play all the video and audio streams of the same resource, realizing a high-performance immersive gaming experience and effectively reducing the latency.

[0103] Among them, both the first server and the second server can be understood as cloud servers.

[0104] The display method provided by the embodiment of the present application is applied to Figure 5A the communication system shown. The difference between this communication system and Figure 1 the communication system shown is that, as Figure 5AIn the communication system shown, the cloud server 110 further includes a server 15, which is used to allocate resources for the terminal and request other servers in the cloud server 110, such as server 11, server 12, server 13, and server 14, to allocate application programs for the terminal. Among them, the cloud server 110 can be understood as a cloud game cluster, which includes multiple servers in the cloud.

[0105] Exemplarily, the above first server can be understood as Figure 5A A server (such as server 15) of the cloud server 110 shown, which is used to communicate with other servers and terminals in the cloud server 110. The first server may include a matching unit and a resource management unit. The matching unit is used to match players who apply for battle matching into the same battle room according to the pre-configured matching rules through the attribute characteristics of the players (such as level, experience). The resource management unit is used to manage resources and allocate resources according to the player matching results. The above second server can be understood as Figure 5A A server (such as server 11, server 12, server 13, or server 14) of the cloud server 110 shown, which is used to communicate with the first terminal; the second server may include a game application instance and a cloud game service unit. Among them, the game application instance is a game application deployed on a virtual mobile phone or virtual PC in the cloud game cluster. After being started, it can provide game interaction operations and screens. The cloud game service unit is responsible for deploying and starting to pull up the game application instance, forwarding the operation instructions sent by the cloud game client to the game application instance, and at the same time forwarding the game audio and video stream to the cloud game client.

[0106] Figure 5B Provided by the embodiments of the present application Figure 5A A schematic flowchart of a display method for the communication system shown. As Figure 5BAs shown in the figure, ① when a first operation on the battle client on the first client of the first terminal is received, the first terminal sends a request to the first server (such as a shared battle server), and this request is used to request matching resources. ② The first server matches the first resource for the first terminal according to this request. Specifically, the matching unit of the first server obtains the first resource from the resource management unit according to the request and in accordance with the matching rules. The first server returns the identification information of the first resource to the first terminal. Similarly, the implementation manner of the second terminal requesting resources from the first server is the same as steps ① and ②, which will not be elaborated here. ③ The first server sends a second request to the second server (such as a cloud server or a cloud game server cluster), and this second request is used to request the allocation of an application (APP) (i.e., a game application instance allocated to each terminal), such as the first APP. The identification information of the first resource and the information of the first terminal are carried in this second request. Of course, the information of the second terminal can also be carried in the second request. Correspondingly, the second server allocates applications to each terminal according to the second request. ④ When a second operation on the second client of the first terminal is received, the first terminal sends an operation instruction corresponding to the second operation to the second server. ⑤ The cloud game service of the second server receives the operation instruction and sends the operation instruction to the first APP (i.e., the game application instance allocated to the first terminal). ⑥ The first APP of the second server executes corresponding operations according to the operation instruction, renders the video and audio streams, and sends the video and audio streams to the cloud game service. ⑦ The cloud game service of the second server sends all the video and audio streams corresponding to the identification information of the first resource to the second client of the first terminal. Similarly, if the resource allocated by the first server to the second terminal is also the first resource, when the user operates on the second client of the second terminal, the executed steps are the same as ④, ⑤, and ⑥. After that, the cloud game service of the second server sends all the video and audio streams corresponding to the identification information of the first resource to the second client of the second terminal.

[0107] It can be seen that, compared with the first method above, the display method provided by the embodiment of the present application configures resources for each terminal through the first server, so that when the second server pushes the video and audio streams, the pushing is based on the resources configured by the first server. In this way, the second server can push all the video and audio streams corresponding to the same resource to all the clients corresponding to this resource, so that the interfaces of other clients can be seen on each client, realizing a high-performance immersive game experience with low latency. In addition, compared with the second method above, it is not necessary to merge and process the video and audio streams of each terminal on the cloud server, reducing the load of the cloud server. Compared with the third method above, game developers only need to embed the client corresponding to the first server, such as a battle client, on the client, and deploy the game application on the cloud server, so that a single-player game can be changed into a battle game at low cost.

[0108] Figure 6 It is a block diagram of the above terminal 120.

[0109] As Figure 20 shown, the terminal 120 may include a processor 111, an external memory interface 122, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0110] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the terminal 120. In other embodiments of the present application, the terminal 120 may include more or fewer components than shown, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0111] The processor 111 may include one or more processing units. For example, the processor 111 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0112] The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0113] A memory can also be set in the processor 111 for storing instructions and data. In some embodiments, the memory in the processor 111 is a cache memory. This memory can store the instructions or data that the processor 111 has just used or recycled. If the processor 111 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 111, and thus improves the efficiency of the system.

[0114] In some embodiments, the processor 111 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0115] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (derail clock line, SCL). In some embodiments, the processor 111 may include multiple groups of I2C buses. The processor 111 can be respectively coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 111 can be coupled to the touch sensor 180K through the I2C interface, so that the processor 111 communicates with the touch sensor 180K through the I2C bus interface to implement the touch function of the terminal 120.

[0116] The I2S interface can be used for audio communication. In some embodiments, the processor 111 may include multiple groups of I2S buses. The processor 111 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 111 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to implement the function of answering a call through a Bluetooth headset.

[0117] The PCM interface can also be used for audio communication to sample, quantize, and encode analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to implement the function of answering a call through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0118] It can be understood that the interface connection relationships between the modules illustrated in the embodiments of the present application are only illustrative and do not constitute a limitation on the structure of the terminal 120. In other embodiments of the present application, the terminal 120 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0119] The charging management module 140 is used to receive a charging input from a charger. Among them, the charger can be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 can receive the charging input from the wired charger through the USB interface 130. In some embodiments of wireless charging, the charging management module 140 can receive the wireless charging input through the wireless charging coil of the terminal 120. While charging the battery 142, the charging management module 140 can also supply power to the electronic device through the power management module 141.

[0120] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 111. The power management module 141 receives the inputs from the battery 142 and / or the charging management module 140 and supplies power to the processor 111, the internal memory 121, the display screen 194, the camera 193, the wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as the battery capacity, the number of battery cycles, and the battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be disposed in the processor 111. In other embodiments, the power management module 141 and the charging management module 140 can also be disposed in the same device.

[0121] The wireless communication function of the terminal 120 can be implemented through Antenna 1, Antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0122] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the terminal 120 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, Antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0123] The mobile communication module 150 can provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the terminal 120. The mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by Antenna 1, filter and amplify the received electromagnetic waves, and then transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through Antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 can be disposed in the processor 111. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 111 can be disposed in the same device.

[0124] The wireless communication module 160 can provide solutions for wireless communications applied to the terminal 120, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves through Antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 111. The wireless communication module 160 can also receive the signals to be transmitted from the processor 111, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through Antenna 2 for radiation.

[0125] In some embodiments, antenna 1 of terminal 120 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling terminal 120 to communicate with the network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0126] Terminal 120 implements the display function through the GPU, display screen 194, and application processor, etc. The GPU is a microprocessor for image processing, connected to display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 111 may include one or more GPUs that execute program instructions to generate or change display information.

[0127] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal 120 may include one or N display screens 194, where N is a positive integer greater than 1.

[0128] The terminal 120 can implement the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc.

[0129] The camera 193 is used to capture still images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard formats such as RGB and YUV. In some embodiments, the terminal 120 may include one or N cameras 193, where N is a positive integer greater than 1.

[0130] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the terminal 120 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0131] The video codec is used to compress or decompress digital videos. The terminal 120 can support one or more video codecs. In this way, the terminal 120 can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0132] The external memory interface 122 can be used to connect an external memory card, such as a Micro SD card, to implement the storage capacity expansion of the terminal 120. The external memory card communicates with the processor 111 through the external memory interface 122 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0133] The internal memory 121 can be used to store computer-executable program codes, and the executable program codes include instructions. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system and application programs required for at least one function (such as the sound playback function, the image playback function, etc.). The data storage area can store the data created during the use of the terminal 120 (such as audio data, phone book, etc.).

[0134] The terminal 120 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor, etc. For example, music playback, recording, etc.

[0135] The audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert analog audio input into a digital audio signal. The audio module 170 can also be used for encoding and decoding audio signals.

[0136] The speaker 170A, also known as the "loudspeaker", is used to convert an audio electrical signal into a sound signal. The terminal 120 can listen to music or hands-free calls through the speaker 170A.

[0137] The receiver 170B, also known as the "earpiece", is used to convert an audio electrical signal into a sound signal. When the terminal 120 answers a call or a voice message, the voice can be listened to by bringing the receiver 170B close to the ear.

[0138] The microphone 170C, also known as the "microphone" or "transmitter", is used to convert a sound signal into an electrical signal. When making a call or sending a voice message, the user can speak by bringing the mouth close to the microphone 170C to input the sound signal into the microphone 170C.

[0139] The headphone jack 170D is used to connect a wired headphone. The headphone jack 170D can be a USB interface 130, or a 3.5 mm open mobile terminal platform (OMTP) standard interface, or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.

[0140] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The terminal 120 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the terminal 120 detects the touch operation intensity according to the pressure sensor 180A. The terminal 120 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0141] The gyroscope sensor 180B can be used to determine the motion posture of the terminal 120. In some embodiments, the angular velocity of the terminal 120 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal 120 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal 120 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0142] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal 120 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0143] The magnetic sensor 180D includes a Hall sensor. The terminal 120 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the terminal 120 is a flip phone, the terminal 120 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover automatic unlocking and other features are set.

[0144] The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal 120 in various directions (generally three axes). When the terminal 120 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0145] A distance sensor 180F is used to measure distance. The terminal 120 can measure distance through infrared or laser. In some embodiments, when shooting a scene, the terminal 120 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0146] The proximity light sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The light-emitting diode may be an infrared light-emitting diode. The terminal 120 emits infrared light outward through the light-emitting diode. The terminal 120 uses the photodiode to detect the infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal 120. When insufficient reflected light is detected, the terminal 120 can determine that there is no object near the terminal 120.

[0147] The ambient light sensor 180L is used to sense the ambient light brightness. The terminal 120 can adaptively adjust the brightness of the display screen 194 according to the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal 120 is in a pocket to prevent accidental touch.

[0148] The fingerprint sensor 180H is used to collect fingerprints. The terminal 120 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint taking pictures, fingerprint answering calls, etc.

[0149] The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal 120 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the terminal 120 reduces the performance of the processor located near the temperature sensor 180J in order to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the terminal 120 heats the battery 142 to avoid abnormal shutdown of the terminal 120 caused by low temperature. In some other embodiments, when the temperature is lower than yet another threshold, the terminal 120 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0150] The touch sensor 180K, also known as the "touch control device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also known as the "touch control screen". The touch sensor 180K is used to detect touch operations acting thereon or nearby. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the terminal 120, at a different position from the display screen 194.

[0151] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals of the vibrating bone mass of the human vocal part. The bone conduction sensor 180M can also contact the human pulse and receive blood pressure pulsation signals.

[0152] The button 190 includes a power-on button, a volume button, etc. The button 190 can be a mechanical button or a touch button. The terminal 120 can receive button inputs and generate key signal inputs related to the user settings and function control of the terminal 120.

[0153] The motor 191 can generate vibration prompts. The motor 191 can be used for incoming call vibration prompts and also for touch vibration feedback. For example, touch operations for different applications (such as taking pictures, audio playing, etc.) can correspond to different vibration feedback effects. For touch operations on different areas of the display screen 194, the motor 191 can also correspond to different vibration feedback effects. Different application scenarios (such as time reminder, receiving messages, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0154] The indicator 192 can be an indicator light and can be used to indicate the charging state, power change, and can also be used to indicate messages, missed calls, notifications, etc.

[0155] The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation from the terminal 120.

[0156] Of course, the terminal 120 can also include other functional units, which are not limited in the embodiments of the present application.

[0157] Next, in combination with the accompanying drawings in the embodiments of the present application, the display method provided by the embodiments of the present application will be described.

[0158] Next, taking Figure 5A and Figure 5B The architecture shown as an example, the display method provided by the embodiments of the present application will be described. Each unit in the following embodiments can have the Figure 5A and Figure 5B Components shown will not be elaborated. It should be noted that the message names or parameter names in the messages exchanged between various devices in the embodiments of the present application are only examples, and other names can also be used in specific implementations. In addition, the actions, terms, etc. involved in the embodiments of the present application can be referred to each other without limitation.

[0159] The following will provide a detailed introduction to a display method provided by the embodiments of the present application.

[0160] Figure 7A This is a schematic flowchart of a display method provided by an embodiment of the present application. As Figure 7A shown, this method is applied to Figure 5A the communication system shown in the figure. The communication system includes a first terminal and a cloud server. This method may include S70 - S74 (where some steps are optional). For specific details, please refer to the description of the following embodiments.

[0161] In S70, when a first operation on the client on the first terminal is received, the first terminal sends a first request to the cloud server. The first request is used to request matching resources.

[0162] In one embodiment, the first terminal has a first client and a second client. Among them, the cloud server may include a first server and a second server. The first server and the second server may be the same server or different servers. The first server corresponds to the first client, and the second server corresponds to the second client. It can be understood that the first client is embedded within the second client. Exemplarily, taking a battle game as an example, the second client is a game center client, and the second client may be a battle client. Of course, the first client and the second client may also use other names, which are not specifically limited herein.

[0163] Exemplarily, when a first operation on the first client on the first terminal is received, the first terminal sends a first request to the cloud server. Specifically, when a first operation on the first client on the first terminal is received, the first terminal sends a first request to the first server. After the first server receives the first request sent by the first terminal, the first server allocates first resources to the first terminal, such as a battle room, the number of players in the battle room, the identification information of the battle room, etc.

[0164] For the specific details of this step, please refer to Figure 7B the relevant descriptions of steps such as S702 in the figure, which will not be elaborated here.

[0165] S71. The first terminal receives the identification information of the first resources returned by the cloud server. The first resources are matched by the cloud server for the first terminal according to the first request.

[0166] In one embodiment, the first request is further used to request the cloud server to match the first resources for the second terminal, and the information of the second terminal is carried in the first request.

[0167] In one embodiment, the information of the first terminal includes the connection address of the first terminal; based on the connection address of the first terminal, a communication connection is established between the first terminal and the cloud server. Specifically, the first terminal sends the connection address of the first terminal to the cloud server, and the cloud server sends a message to the first terminal according to the connection address of the first terminal. In this way, a communication connection is directly established between the first terminal and the cloud server.

[0168] For the specific details of this step, please refer to Figure 7B the relevant descriptions of steps such as S706 in

[0169] S72. When a second operation on the client on the first terminal is received, the first terminal sends a first message to the cloud server. The first message includes the operation instruction corresponding to the second operation, the information of the first terminal, and the identification information of the first resource.

[0170] In one embodiment, the first terminal has a first client and a second client. When a second operation on the second client on the first terminal is received, the first terminal sends a first message to the cloud server. Specifically, when a second operation on the second client on the first terminal is received, the first terminal sends a first message to the second server. The second server performs the operation corresponding to the operation instruction and renders the video and audio stream. The rendered video and audio stream has a mapping relationship with the identification of the resource.

[0171] For the specific details of this step, please refer to Figure 7B the relevant descriptions of steps such as S711 in

[0172] S73. The first terminal receives all the video and audio streams corresponding to the identification information of the first resource returned by the cloud server. The video and audio streams are rendered by the cloud server.

[0173] For the specific details of this step, please refer to Figure 7B the relevant descriptions of steps such as S713 in

[0174] S74. The first terminal displays a first interface, which is formed by all the video and audio streams corresponding to the identification information of the first resource.

[0175] In one embodiment, when the first terminal displays the first interface, specifically: the first terminal displays the first interface according to a preset display rule. Further, the preset display rule includes any one of the following: a split-screen display rule, which is a rule for dividing the display area of the display screen according to a preset ratio; a screen-switching display rule, which is a rule for switching the content displayed in the display area of the display screen after the terminal receives a screen-switching operation.

[0176] For the specific details of this step, please refer to Figure 7BThe relevant descriptions of steps such as S714 will not be elaborated here.

[0177] In the embodiments of the present application, the first server configures resources for each terminal, so that when the second server pushes video and audio streams, the pushing is based on the resources configured by the first server. In this way, the second server can push all video and audio streams corresponding to the same resource to all clients corresponding to this resource, so that each client can see the interfaces of other clients, achieving a high-performance immersive gaming experience with low latency. In addition, it is not necessary to merge and process the video and audio streams of each terminal on the cloud server, reducing the load on the cloud server. In addition, game developers only need to embed the client corresponding to the first server, such as a battle client, on the client, and deploy the game application on the cloud server, so that a single-player game can be changed into a battle game at low cost.

[0178] The following provides a detailed introduction to a display method provided by the embodiments of the present application.

[0179] Figure 7B It is a schematic flowchart of a display method provided by the embodiments of the present application. As Figure 7B shown, this method is applied to Figure 5A the communication system shown, and this method may include S700 - S714 (where some steps are optional), and for details, please refer to the descriptions of the following embodiments.

[0180] In S700, in response to receiving an operation 1 of the user on the first client (such as the second client above) on the first interface, the first terminal displays the second interface, and this second interface is the login interface of the first client.

[0181] Among them, the operation 1 of the user may include a click operation, a swipe operation, a press operation, etc. This click operation may include a single - click operation, a double - click operation, etc.

[0182] When the user needs to log in to the first client, the user clicks the icon of the first client. The first terminal receives the operation of the user, and in response to this operation, the first terminal displays the login interface of the first client. For example, assume that the first client is a cloud game client, such as Figure 8 the game center 102 on the interface 101 of the first terminal shown. When the user clicks the icon of the game center 102, in response to this operation, the first terminal performs the operation of logging in to the game center 102. At this time, the first terminal displays the second interface, that is, the login interface of the cloud game client (such as Figure 9 the interface 201 shown).

[0183] S701. When the first terminal receives the second operation of the user on the second interface, the first terminal displays the third interface, which is the interface after logging in to the first client. The third interface includes icons of multiple APPs and the user's friend information.

[0184] Among them, the second operation of the user may include an input operation, etc. For example, an operation of inputting a user account and password.

[0185] After the user inputs the user account and password on the login interface of the cloud game client, in response to this operation, the first terminal displays the interface of the first client, on which the user information, the user's friend information, and icons of multiple APPs are displayed. For example, assume that the multiple APPs are multiple cloud game APPs. The user Figure 9 inputs the user account and password (such as Figure 10 the user account shown is player A and the password is ******) on the login interface of the first client (such as the cloud game client) as shown. After that, the user clicks Figure 10 the control 202 on the interface 201 as shown. The first terminal receives the operation of the user, and in response to this operation, the interface of the first terminal jumps from Figure 10 the interface 201 as shown to Figure 11 the interface 301 as shown. The interface 301 displays the user information (such as player A), the user's friend information (such as player B, player C, and player D), and icons of multiple APPs (such as cloud game 1, cloud game 2, cloud game 3, and cloud game 4, etc.).

[0186] S702. When the first terminal receives the third operation (such as the first operation above) of the user on the third interface for the icon of the first APP and the friend information, the first client of the first terminal calls the second client (such as the first client above) to send a first request (such as the first request above) to the first server of the cloud server. The first request is used to invite friends to participate in the game battle of the first APP, and the user information and the user's friend information are carried in the first request. Correspondingly, the first server receives the first request.

[0187] Among them, the second operation of the user is the same as the first operation above. Of course, the second operation of the user may also include other forms of selection operations. For example, an operation of sliding the cursor over the area where the first APP and the friend information are located, or an operation of moving the cursor to the icon of the first APP and the friend information, etc. The embodiments of the present application do not make specific limitations.

[0188] In the embodiments of the present application, the operations of the user on the first APP on the third interface can be understood as the following situations: First, the operation of the user on the area where the icon of the first APP is located on the third interface. Second, the triggering operation on the display interface of the icon of the first APP, in other words, the triggering operation on the display interface and not on the icon of the first APP. Of course, there can be other situations, which will not be listed one by one here.

[0189] In addition, the operation on the first APP can be the user's direct operation on the first APP, such as the user's finger clicking on the icon of the first APP or the first control corresponding to the first APP. It can also be the user's indirect operation on the first APP, such as the user moving the cursor of the mouse to the icon of the first APP.

[0190] When the user needs to start the first APP among multiple APPs. On the interface of the first client, the user selects friends to participate in the cloud game corresponding to the first APP, and selects the first APP to start. For example, the user selects the first APP (such as cloud game 1) among the icons of multiple APPs, and the user sequentially selects friends (such as player B, player C, and player D).

[0191] S703. According to Request 1, the first server sends an invitation message to the second client of the user's friend (such as the first client of the second terminal mentioned above). Correspondingly, the second client of the user's friend receives the invitation message.

[0192] The invitation message may include information about the inviting player, information about the invited player, and the invitation event. For example, the inviting player may include the user (i.e., player A), and the information about the invited player may include information about player B, information about player C, or information about player D. The invitation event may be to play a cloud game corresponding to the first APP.

[0193] Further, when the user selects a friend, a control for indicating an invitation to play a game is displayed in the corresponding area where the friend information is displayed. When the user clicks on this control, the first terminal sends a request to the first server to invite the friend to participate in the game battle of the first APP. For example, as Figure 12 shown, when the user selects player B, a control 302 is displayed in the corresponding area where player B is displayed. This control 302 can be expressed as an invitation for player B to play a game. When the user clicks on control 302, the terminal sends a request to the first server, and this request is a request for invitation.

[0194] S704. The second client of the user's friend displays the fourth interface, and on this fourth interface, the user information, the user's invitation information, and a control for confirmation are displayed.

[0195] Exemplarily, after the first server sends an invitation message to the client of Player B according to the request, the information of the user (Player A) and the invitation information of Player A are displayed on the client of Player B. A control for determination is also displayed on the client of Player B. For example, Figure 13 As shown in the interface 601 of the client of Player B, the information of Player A (the username of Player A), the invitation information of Player A (such as "Let's play a game") and the controls 602 and 603 are displayed on this interface 601. The control 602 is a control for indicating agreement, such as the "Readily Agree" control. The control 603 is a control for indicating disagreement, such as the "Cruelly Refuse" control. Similarly, for Player A to invite other friends (such as Player C and Player D) to participate in the game battle of the first APP, reference can be made to the above description and will not be elaborated here.

[0196] S705. When receiving the operation four of the user's friend on Interface Four, the second terminal sends Request Two (such as the above-mentioned First Request and Third Request) to the first server, and this Request Two is used to request a battle match. Correspondingly, the first server receives Request Two.

[0197] In a specific implementable manner, refer to Figure 5B Step ① shown in and will not be elaborated here.

[0198] Among them, the operation four is the operation of the user's friend on the client of their terminal. For example, the operation on the control on the client. Exemplarily, the control is the "Readily Agree" control.

[0199] Continuing with the above example, assume terminal A corresponding to player A's client, terminal B corresponding to player B's client, terminal C corresponding to player C's client, and terminal D corresponding to player D's client. When a friend of player A clicks the "Readily Agree" control, terminal A corresponding to player A's client sends request two to the first server. This request two carries player A's identification information and game information. For example, player A's identification information is PlayerId-A. Player A's game information may include player A's game character level. For example, player A's game character level is level 10. Similarly, when a friend of player B clicks the "Readily Agree" control, terminal B corresponding to player B's client sends request two to the first server. This request two carries player B's identification information and game information. For example, player B's identification information is PlayerId-B. Player B's game information may include player B's game character level. For example, player B's game character level is level 11. When a friend of player C clicks the "Readily Agree" control, terminal C corresponding to player C's client sends request two to the first server. This request two carries player C's identification information and game information. For example, player C's identification information is PlayerId-C. Player C's game information may include player C's game character level. For example, player C's game character level is level 12. When a friend of player D clicks the "Readily Agree" control, terminal D corresponding to player D's client sends request two to the first server. This request two carries player D's identification information and game information. For example, player D's identification information is PlayerId-D. Player D's game information may include player D's game character level. For example, player D's game character level is level 15.

[0200] S706. The first server sends matching information to each terminal according to request two and the matching rule. This matching information carries the first resource, which is allocated by the server for each player. Correspondingly, each terminal receives the matching information.

[0201] In a specific implementable manner, refer to Figure 5B step ② shown in

[0202] Among them, the matching rule can be used to match the game information of each player. This matching rule can be pre-configured by the first server.

[0203] Exemplarily, this matching rule can be that those with game character levels in the first numerical range can be matched into a battle situation. For example, those with game character levels between 0 - 10 can be matched into a battle situation, and those with game character levels between 10 - 100 can be matched into a battle situation.

[0204] Exemplarily, the representation code of the matching rule can be:

[0205]

[0206]

[0207] Among them, "teams" represents the parameters of the participating competing teams, "minPlayer" represents the maximum number of members in the team, "maxPlayer" represents the minimum number of members, "number" represents the number of competing teams, "playerParams" represents the matching parameters calculated by the matching engine, "name" represents the parameter name, "type" represents the parameter type, and "mathTeamRules" represents that the matching rules are parsed and executed by the engine. The matching rules are configured with a scope value rule, using the "teams.players.level" parameter. When the parameter range is between 0 and 10, a competing game can be matched; when the parameter range is between 10 and 100, a competing game can be matched.

[0208] Continuing with the above example, the game character levels of Player A, Player B, Player C, and Player D are 10, 11, 12, and 15 respectively, which are within the parameter range of 10 - 100. Then, Player A, Player B, Player C, and Player D can be matched into a competing game. At this time, the first server allocates first resources to Player A, Player B, Player C, and Player D. Such first resources can include, for example, the identification information of the game competing room (such as RoomId - 001) and the number of players in the competing room. In this way, the matching information is a successful match, and the matching information carries the first resources allocated by the first server to each player.

[0209] S707. The first server sends Request Three (such as the above - mentioned second request) to the second server. This Request Three is used to request the allocation of an application program, and the identification information of the first resources and the information of each terminal are carried in the Request Three. Correspondingly, the second server receives Request Three.

[0210] In a specific implementable manner, refer to Figure 5B Step ③ shown in

[0211] S708. The second server allocates an application program (such as the first APP) to each terminal according to Request Three. The first APP of the second server sends the identification information of the application program to each terminal. Correspondingly, each terminal receives the identification information of the application program.

[0212] In a specific implementable manner, refer to Figure 5B Step ③ shown in

[0213] Specifically, the second server can establish the correspondence relationship among the identification information of the first resource, the information of each terminal, and the application program according to Request 3. Exemplarily, the second server associates the ID of the first resource, the IDs of each terminal, and the first APP.

[0214] S709. Each terminal displays Interface Five, which is a game battle prompt interface.

[0215] Exemplarily, after each terminal corresponding to the client of each player receives the matching information sent by the server, each terminal displays Interface Five, and the game battle prompt information of each player is displayed on Interface Five. For example, as Figure 14 shown in the schematic diagram of the interface corresponding to each player's perspective.

[0216] This game battle prompt interface can be understood as a game loading interface. That is to say, each terminal displays the game loading interface and is in a state of waiting to enter the game. After the game is loaded successfully, the game interface is displayed on the interface of each terminal. The players corresponding to each terminal can operate on the game interface. When each player operates on the game interface of their respective terminal, S708 is executed. Specifically as follows:

[0217] S710. When receiving the interface operation on each terminal for Client One (such as the above-mentioned second client), each terminal sends Request 4 to the second server. This Request 4 is used to request to establish a communication channel, and the connection address (such as the IP address) of each terminal and the identification information of the first resource are carried in this Request 4. Correspondingly, the second server receives Request 4 and establishes a communication connection with each terminal.

[0218] Specifically, when receiving the interface operation of the user on Client One on the first terminal, the first terminal sends a request to the second server, and the connection address of the first terminal is carried in this request. In this way, after the first terminal sends the connection address (such as the IP address) of the first terminal to the second server, subsequently, the second server can push the video and audio stream to the first terminal according to the connection address of the first terminal. Similarly, when receiving the interface operation of the user's friend on Client One on the second terminal, the second terminal sends a request to the second server, and the connection address of the second terminal is carried in this request. In this way, after the second terminal sends the connection address (such as the IP address) of the second terminal to the second server, subsequently, the second server can push the video and audio stream to the second terminal according to the connection address of the second terminal.

[0219] After each terminal establishes a communication connection with the second server, each terminal and the second server can communicate. Specifically, each terminal can send an operation instruction to the second server, and the second server can send a video and audio stream to each terminal. For details, see S711 - S713.

[0220] S711. When receiving the interface operation on each terminal for Client 1 (such as the above-mentioned second operation), each terminal sends an operation instruction to the second server. Correspondingly, the second server receives the operation instruction.

[0221] In a specific implementable manner, refer to Figure 5B Step ④ shown in, which will not be elaborated here.

[0222] Specifically, the interface operation of the user on the first terminal for Client 1 can refer to the operation during the cloud game process of the user. In response to this operation, the first terminal sends an operation instruction to the second server. Similarly, the interface operation of the user's friend on the second terminal for the second client can refer to the operation during the cloud game process of the user. In response to this operation, the second terminal sends an operation instruction to the second server.

[0223] That is to say, the terminals of each player all send operation instructions to the second server. Correspondingly, the second server receives the operation instructions of multiple players under the same resource.

[0224] S712. The second server executes the operation corresponding to the operation instruction according to the operation instruction and renders the video and audio stream.

[0225] After the second server receives the operation instructions of multiple players under the same resource, the second server executes the corresponding operations according to these operation instructions and renders the video and audio streams of each player. Exemplarily, the second server can collect the operation instructions uploaded by each terminal according to the identification information of the game battle room and render the video and audio streams of the terminals of each player.

[0226] S713. The second server pushes all the video and audio streams corresponding to the same resource (such as the above-mentioned first resource) to each terminal of each player that matches the same resource. Correspondingly, each terminal receives the video and audio stream pushed by the second server.

[0227] Exemplarily, the second server pushes the video and audio stream corresponding to the identification information of the game battle room to the terminals of each player corresponding to the identification information according to the identification information of the game battle room.

[0228] S714. The screen of each terminal plays the audio and video, and each terminal displays Interface 6 (such as the above-mentioned first interface) according to the display rule, and the interface shows the interface formed by the audio and video streams of each player.

[0229] Specifically, after the first client of the first terminal receives all the video and audio streams corresponding to the identification information of the game battle room, the screen of the first terminal plays the audio and video. At this time, the first terminal displays Interface Six on the display interface according to the display rules. Similarly, after the second client of the second terminal receives all the video and audio streams corresponding to the identification information of the game battle room, the screen of the second terminal plays the audio and video. At this time, the second terminal displays Interface Six on the display interface according to the display rules. That is to say, after each player's terminal receives all the video and audio streams corresponding to the identification information of the game battle room, the screen of each terminal plays the audio and video of all players.

[0230] Among them, the display rules can include split-screen display rules and screen-switching display rules. The following is a detailed description for different display rules:

[0231] First, the split-screen display rules.

[0232] Among them, the split-screen display rules can be the rules for equally dividing the display screen according to a preset ratio, and the split-screen rules with a preset proportion. Among them, the equal split-screen display rule means that the display area of the display screen is evenly distributed according to the number of players. Or, the display area of the display screen is equally divided according to a preset ratio. For example, area A accounts for 1 / 2 of the entire display area, and area B accounts for 1 / 4 of the entire display area. The split-screen rule with a preset proportion means that the display area of the display screen is distributed according to the preset ratio and the number of players. Among them, the preset ratio / preset proportion can be set according to the actual situation and is not specifically limited here.

[0233] Exemplarily, as Figure 15 shown, taking the equal split-screen display rule as an example, assume that: the interface formed by the video and audio stream of player A is Interface a, and player A's information and player A's game interface are displayed on this Interface a; the interface formed by the video and audio stream of player B is Interface b, and player B's information and player B's game interface are displayed on this Interface b; the interface formed by the video and audio stream of player C is Interface c, and player C's information and player C's game interface are displayed on this Interface c; the interface formed by the video and audio stream of player D is Interface d, and player D's information and player D's game interface are displayed on this Interface d. Taking player A's terminal as an example, Interface a, Interface b, Interface c, and Interface d are displayed on player A's terminal. These interfaces evenly distribute the display area on the display screen of player A's terminal.

[0234] It can be seen that the terminal uses the split-screen display rule to display the interfaces formed by the video and audio streams of each terminal, which is more suitable for large-screen terminals such as Huawei Smart Screens. Through the split-screen display method, it brings a low-latency and high-performance immersive game experience to game players and audiences.

[0235] Second, the screen-switching display rules.

[0236] The screen switching display rule may include switching the content displayed in the display area after the terminal receives a screen switching operation. Exemplarily, taking the first terminal as an example, the display interface of the first terminal includes a first display area and a second display area, and the first display area is larger than the second display area. Further, the display attribute of the second display area may be floating display. The display interface of the first terminal further includes a third display area, which is used to display a player list. The player list may be displayed in a reduced size in the form of player avatars. When the user clicks on the player avatar in the player list, the interface formed by the video and audio stream corresponding to the player may be displayed in the first display area or the second display area.

[0237] Exemplarily, as Figure 16 shown, taking the terminal of player A as an example, on the first display area of the terminal of player A, the interface a formed by the video and audio stream of player A is displayed, and on the second display area, the interface b formed by the video and audio stream of player B is displayed. The player avatars of other players (such as player C and player D) are displayed in the third display area. When clicking on the Figure 17 second display area as Figure 17 shown, the display interface of the terminal of player A jumps from the Figure 18 interface 901 as Figure 19 shown to the Figure 19 interface 902 as Figure 20 shown. On this interface 902, the interface b formed by the video and audio stream of player B is displayed in the first display area, and the interface a formed by the video and audio stream of player A is displayed in the second display area. When clicking on the avatar of player C in the Figure 19 third display area as Figure 19 shown, the display interface of the terminal of player A jumps from the Figure 20 interface 901 as

[0238] interface 903 as Figure 20 shown. On this interface 903, the interface a formed by the video and audio stream of player A is displayed in the first display area, the interface c formed by the video and audio stream of player C is displayed in the second display area, and the player avatars of other players (such as player B and player D) are displayed in the third display area. Of course, there may be other switching forms, which will not be listed one by one here.

[0238] It can be seen that the terminal adopts the screen switching display rule to display the interfaces formed by the video and audio streams of each terminal, which is more suitable for terminals with a smaller display area, such as mobile phones and tablets, thus enhancing the screen switching cloud game battle experience on terminals with a smaller display area such as mobile phones and tablets.

[0239] Under the premise of no contradiction, the various solutions in the above embodiments of the present application can be combined.

[0240] It can be understood that, in order to implement the above functions, the terminal includes the corresponding hardware and / or software modules for executing each function. Combining the algorithm steps of each example described in the embodiments disclosed in this article, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to exceed the scope of the present application.

[0241] In this embodiment, the terminal can be divided into functional modules according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0242] For example, in one division method, referring to Figure 21 , the terminal 2100 may include: a sending module 2101, a receiving module 2102, and a display module 2103. Among them,

[0243] The sending module 2101 is used to send a first request to the cloud server when receiving a first operation on the client of the first terminal, and the first request is used to request matching resources;

[0244] The receiving module 2102 is used to receive the identification information of the first resource returned by the cloud server, and the first resource is matched by the cloud server for the first terminal according to the first request;

[0245] The sending module 2101 is used to send a first message to the cloud server when receiving a second operation on the client of the first terminal, and the first message includes the operation instruction corresponding to the second operation, the information of the first terminal, and the identification information of the first resource;

[0246] The receiving module 2102 is used to receive all the video and audio streams corresponding to the identification information of the first resource, and the video and audio streams are rendered by the cloud server;

[0247] The display module 2103 is used to display a first interface, and the first interface is formed by all the video and audio streams corresponding to the identification information of the first resource.

[0248] In a specific implementable manner, the first terminal has a first client and a second client,

[0249] The sending module 2101 is configured to send a first request to the cloud server when receiving a first operation on a first client of the first terminal; and send a first message to the cloud server when receiving a second operation on a second client of the first terminal.

[0250] In some implementable manners, the communication system further includes a second terminal, and the second terminal has a first client; the first request is further configured to request the cloud server to match a first resource for the second terminal, and information of the second terminal is carried in the first request.

[0251] In some implementable manners, the information of the first terminal includes a connection address of the first terminal; and further includes: a establishing module 2104, configured to establish a communication connection between the first terminal and the cloud server based on the connection address of the first terminal.

[0252] In a specific implementable manner, the display module 2103 is configured to: display a first interface according to a preset display rule.

[0253] In a specific implementable manner, the preset display rule includes any one of the following: a split-screen display rule, which is a rule for dividing the display area of the display screen according to a preset ratio; a screen-switching display rule, which is a rule for switching the content displayed in the display area of the display screen after the terminal receives a screen-switching operation.

[0254] Wherein, the display module 2103 may be a display, for example, specifically may be Figure 6 the display 192 in the shown hardware structure. Exemplarily, the display module 2103 specifically executes Figure 7B the steps S700 - S702, S709 and S714 shown therein. The sending module 2101 and the receiving module 2102 may be devices for interacting with other electronic devices such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc. For example, the sending module 2101 and the receiving module 2102 may include Figure 6 the wireless communication module 160, the antenna 2, etc. in the shown hardware structure. Exemplarily, the sending module 2101 and the receiving module 2102 specifically execute Figure 7B the steps S702, S705, S710, S711 and S713 shown therein.

[0255] It should be noted that all relevant contents of the steps involved in the above method embodiments can be cited in the function descriptions of the corresponding functional modules, and will not be elaborated herein.

[0256] The embodiment of the present application further provides a system, and the system may include: a first terminal and a cloud server;

[0257] The first terminal is used to perform: when receiving a first operation on the client on the first terminal, send a first request to the cloud server, where the first request is used to request matching resources.

[0258] The cloud server is used to perform: receive the first request; match a first resource for the first terminal according to the first request; return the identification information of the first resource to the first terminal.

[0259] The first terminal is further used to perform: receive the identification information of the first resource; when receiving a second operation on the client on the first terminal, send a first message to the cloud server, where the first message includes an operation instruction corresponding to the second operation, the information of the first terminal, and the identification information of the first resource.

[0260] The cloud server is further used to perform: render a video and audio stream according to the first message; send all video and audio streams corresponding to the identification information of the first resource to the first terminal.

[0261] The first terminal is further used to perform: receive all video and audio streams corresponding to the identification information of the first resource; display a first interface, where the first interface is formed by all video and audio streams corresponding to the identification information of the first resource.

[0262] In a specific implementable manner, the cloud server includes a first server and a second server;

[0263] The first server is used to perform: receive the first request sent by the first terminal, match a first resource for the first terminal according to the first request, and return the identification information of the first resource to the first terminal;

[0264] The second server is used to perform: receive the first message sent by the first terminal, render a video and audio stream according to the first message, and send all video and audio streams corresponding to the identification information of the first resource to the first terminal.

[0265] In a specific implementable manner, the first server is further used to perform: send a second request to the second server, where the second request is used to request to allocate an application program for the first terminal, and the second request carries the identification information of the first resource and the information of the first terminal.

[0266] The second server is further used to perform: allocate an application program for the first terminal according to the second request.

[0267] In a specific implementable manner, the second server is further used to perform: establish an association relationship among the identification information of the first resource, the information of the first terminal, and the application program.

[0268] In a specific implementable manner, the first terminal has a first client and a second client. The first client corresponds to the first server, and the second client corresponds to the second server. The first terminal is further configured to perform:

[0269] When receiving a first operation on the first client on the first terminal, send a first request to the first server;

[0270] When receiving a second operation on the second client on the first terminal, send a first message to the second server.

[0271] In some implementable manners, the communication system further includes a second terminal. The second terminal has a first client and a second client. The first terminal is further configured to perform: when receiving a first operation on the first client on the first terminal, send a third request to the first server, and the third request is used to request to invite the first client of the second terminal;

[0272] The first server is further configured to perform: match a first resource for the second terminal according to the third request.

[0273] In a specific implementable manner, the information of the first terminal includes the connection address of the first terminal. The first terminal is further configured to perform: based on the connection address of the first terminal, establish a communication connection between the first terminal and the cloud server.

[0274] In a specific implementable manner, the first terminal is further configured to perform: display a first interface according to a preset display rule.

[0275] In a specific implementable manner, the preset display rule includes any one of the following: a split-screen display rule, which is a rule for dividing the display area of the display screen according to a preset ratio; a screen-switching display rule, which is a rule for switching the content displayed in the display area of the display screen after the terminal receives a screen-switching operation.

[0276] The embodiment of the present application further provides a computer-readable storage medium, including instructions, which when running on a computer, cause the computer to execute any of the above methods.

[0277] The embodiment of the present application further provides a computer program product containing instructions, which when running on a computer, cause the computer to execute any of the above methods.

[0278] The embodiment of the present application further provides a chip, which includes a processor and an interface circuit. The interface circuit is coupled to the processor. The processor is used to run a computer program or instructions to implement the above method, and the interface circuit is used to communicate with other modules outside the chip.

[0279] In the description of this application, unless otherwise specified, " / " means "or". For example, A / B can mean A or B. "And / or" in this article is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, "at least one" means one or more, and "a plurality" means two or more. The words such as "first" and "second" do not limit the quantity and execution order, and the words such as "first" and "second" do not necessarily limit that they are different.

[0280] In the description of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific manner.

[0281] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0282] In several embodiments provided in this application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0283] The unit described as a separate component may or may not be physically separated. The component displayed as a unit may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0284] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0285] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0286] The above content is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A display method, applied to a communication system, the communication system comprising a first terminal and a cloud server, characterized in that, The cloud server includes a first server and a second server. The first server is used to match resources for the first terminal and to request the second server to allocate a game instance to the first terminal. The second server is used to allocate game resources to the first terminal and to send the audio-video streams of the same resource to all clients corresponding to the same resource. The method includes: When a first operation on the client on the first terminal is received, the first terminal sends a first request to the first server, and this first request is used to request resource matching. The first terminal receives the identification information of the first resource returned by the first server, and the first resource is matched for the first terminal by the first server according to the first request. When a second operation on the client on the first terminal is received, the first terminal sends a first message to the second server, and the first message includes the operation instruction corresponding to the second operation, the information of the first terminal, and the identification information of the first resource. The first terminal receives all the video and audio streams corresponding to the identification information of the first resource returned by the second server. All the audio-video streams include the audio-video streams of all clients corresponding to the first resource, and the video and audio streams are rendered by the second server. The first terminal displays a first interface, and the first interface is formed by all the video and audio streams corresponding to the identification information of the first resource.

2. The method according to claim 1, wherein The first terminal has a first client and a second client. The first operation acts on the first client on the first terminal; the second operation acts on the second client. When a first operation on the first client on the first terminal is received, the first terminal sends the first request to the cloud server. When a second operation on the second client on the first terminal is received, the first terminal sends the first message to the cloud server.

3. The method according to claim 1 or 2, characterized in that, The communication system further includes a second terminal, and the second terminal has a first client and a second client; the information of the second terminal is carried in the first request; the first request is further used to request the first server to match resources for the second terminal. The first client on the second terminal is used to receive the identification information of the first resource returned by the cloud server. The second client on the second terminal is used to receive a third operation, send a second message to the cloud server, and display a second interface. The second message includes the operation instruction corresponding to the third operation, the information of the second terminal, and the identification information of the first resource.

4. The method according to claim 2, wherein The information of the first terminal includes the connection address of the first terminal. Based on the connection address of the first terminal, a communication connection is established between the first terminal and the cloud server.

5. The method according to claim 1, wherein The first terminal displays the first interface, including: The first terminal displays the first interface according to a preset display rule.

6. The method according to claim 5, characterized in that, The preset display rule includes any one of the following: A split-screen display rule, and the split-screen display rule is a rule for dividing the display area of the display screen according to a preset ratio. The screen switching display rule is a rule for switching the content displayed in the display area of the display screen after the terminal receives a screen switching operation.

7. A display method, applied to a communication system, the communication system including a first terminal and a cloud server, characterized in that, The cloud server includes a first server and a second server. The first server is used to match resources for the first terminal and to request the second server to request the allocation of a game instance to the first terminal. The second server is used to allocate game resources to the first terminal and to send the audio-video streams of the same resource to all clients corresponding to the same resource. The method includes: When a first operation on the client on the first terminal is received, the first terminal sends a first request to the first server. This first request is used to request resource matching. The first server receives the first request. The first server matches a first resource for the first terminal according to the first request. The first server returns the identification information of the first resource to the first terminal. The first terminal receives the identification information of the first resource. When a second operation on the client on the first terminal is received, the first terminal sends a first message to the second server. The first message includes the operation instruction corresponding to the second operation, the information of the first terminal, and the identification information of the first resource. The second server renders the video and audio streams according to the first message. The second server sends all the video and audio streams corresponding to the identification information of the first resource to the first terminal. All the audio-video streams include the audio-video streams of all clients corresponding to the first resource. The first terminal receives all the video and audio streams corresponding to the identification information of the first resource. The first terminal displays a first interface, which is formed by all the video and audio streams corresponding to the identification information of the first resource.

8. The method according to claim 7, wherein It further includes: The first server sends a second request to the second server. The second request is used to request the allocation of an application program for the first terminal. The identification information of the first resource and the information of the first terminal are carried in the second request. The second server allocates an application program for the first terminal according to the second request.

9. The method according to claim 8, characterized in that It further includes: The second server establishes an association relationship among the identification information of the first resource, the information of the first terminal, and the application program.

10. The method according to claim 7, wherein The first terminal has a first client and a second client. The first client corresponds to the first server, and the second client corresponds to the second server. When a first operation on the first client on the first terminal is received, the first terminal sends the first request to the first server. When a second operation on the second client on the first terminal is received, the first terminal sends the first message to the second server.

11. The method according to claim 7, wherein The communication system further includes a second terminal. The second terminal has a first client and a second client. When a first operation on a first client on the first terminal is received, the first terminal sends a third request to a first server in the cloud server, and the third request is used to request to invite a first client of the second terminal; The first server matches the first resource for the second terminal according to the third request.

12. The method according to claim 7, characterized in that The information of the first terminal includes the connection address of the first terminal; it further includes: Based on the connection address of the first terminal, a communication connection is established between the first terminal and the cloud server.

13. The method according to claim 7, wherein The first terminal displays a first interface, including: The first terminal displays the first interface according to a preset display rule.

14. The method according to claim 13, wherein The preset display rule includes any one of the following: A split-screen display rule, and the split-screen display rule is a rule for dividing the display area of the display screen according to a preset ratio; A screen-switching display rule, and the screen-switching display rule is a rule for switching the content displayed in the display area of the display screen after the terminal receives a screen-switching operation.

15. A terminal, the terminal includes a touch screen, a memory, a transceiver, one or more processors, a plurality of applications, and one or more programs; wherein, The one or more programs are stored in the memory, and the transceiver is used to transmit or receive wireless signals; characterized in that when the one or more processors execute the one or more programs, the terminal implements the method according to any one of claims 1-6.

16. A communication system, characterized in that, The system includes: a first terminal and a cloud server; the cloud server includes a first server and a second server, the first server is used to match resources for the first terminal, and is used to request the second server to request to allocate a game instance to the first terminal; the second server is used to allocate game resources to the first terminal, and send the audio-video streams of the same resource to all clients corresponding to the same resource; The first terminal is used to execute: When a first operation on a client on the first terminal is received, send a first request to the first server, and the first request is used to request to match resources; The first server is used to execute: Receive the first request; According to the first request, match a first resource for the first terminal; Return the identification information of the first resource to the first terminal; The first terminal is further used to execute: Receive the identification information of the first resource; When a second operation on a client on the first terminal is received, send a first message to the second server, and the first message includes an operation instruction corresponding to the second operation, the information of the first terminal, and the identification information of the first resource; The second server is further used to execute: Render video and audio streams according to the first message; Send all video and audio streams corresponding to the identification information of the first resource to the first terminal; all the audio-video streams include the audio-video streams of all clients corresponding to the first resource; The first terminal is further used to execute: Receive all video and audio streams corresponding to the identification information of the first resource; Display a first interface, and the first interface is formed by all video and audio streams corresponding to the identification information of the first resource.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when run on a terminal, cause the terminal to perform the method according to any one of claims 1-6.

18. A chip system, characterized in that, Comprising one or more processors that, when the one or more processors execute the instructions, perform the method according to any one of claims 1-6.

Images