Virtual game processing method and device, electronic equipment and storage medium

By displaying the player's screen and camera image collection in the virtual game, providing comprehensive monitoring information is solved, and the problem of fairness monitoring of virtual games is achieved, and the referee's effective penalty support is achieved.

CN119075307APending Publication Date: 2024-12-06TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310658787.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-05
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing technology is difficult to effectively monitor and ensure the fairness of virtual games, and there is a lack of effective referee support tools.

Method used

By displaying monitoring controls in the human-computer interactive interface, multiple players' pictures are displayed in response to trigger operations, including screen image acquisition and camera image acquisition, providing all-round information to support referee judgments.

Benefits of technology

Multi-dimensional monitoring of all players in the virtual game is achieved, sufficient penalty basis is provided, and the fairness of the virtual game is ensured.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119075307A_ABST
    Figure CN119075307A_ABST
Patent Text Reader

Abstract

The invention provides a virtual game processing method and device, electronic equipment and a storage medium. The method comprises the following steps: displaying a monitoring control in a human-computer interaction interface; in response to a trigger operation for the monitoring control, displaying pictures corresponding to a plurality of players included in at least one virtual game, the types of the pictures comprise a first picture obtained by carrying out image acquisition on a screen of the electronic equipment used by the player and a second picture obtained by carrying out image acquisition on the player. According to the method and the device, all-around information can be provided for the referees of the virtual game, the referees can watch and judge conveniently, and thus the fairness of the virtual game is guaranteed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer human-computer interaction technology, and in particular to a method, device, electronic device and storage medium for processing a virtual game. Background Art

[0002] Display technology based on graphics processing hardware has expanded the channels for perceiving the environment and obtaining information, especially the display technology of virtual scenes. It can realize diversified interactions between virtual objects controlled by users or artificial intelligence according to actual application needs, and has various typical application scenarios. For example, in the virtual scenes of games, it can simulate the real battle process between virtual objects.

[0003] Taking competitive games as an example, as competitive games become more popular, more and more game events based on competitive games are being held. However, there is no effective solution in the relevant technology for monitoring game events to ensure the fairness of game events. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device, computer-readable storage medium and computer program product for processing a virtual game, which can provide all-round information to the referee of the virtual game, facilitate the referee to watch and judge, thereby ensuring the fairness of the virtual game.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides a method for processing a virtual game, including:

[0007] Display monitoring controls in the human-computer interaction interface;

[0008] In response to a trigger operation on the monitoring control, screens corresponding to a plurality of players included in at least one virtual game are displayed, wherein the types of the screens include: a first screen obtained by capturing an image of a screen of an electronic device used by the player, and a second screen obtained by capturing an image of the player.

[0009] The embodiment of the present application provides a virtual game processing device, including:

[0010] A display module, used to display monitoring controls in a human-computer interaction interface;

[0011] The display module is further used to display screens corresponding to multiple players included in at least one virtual game in response to a trigger operation on the monitoring control, wherein the types of the screens include: a first screen obtained by capturing an image of a screen of an electronic device used by the player, and a second screen obtained by capturing an image of the player.

[0012] An embodiment of the present application provides an electronic device, including:

[0013] A memory for storing executable instructions;

[0014] The processor is used to implement the virtual game processing method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0015] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for implementing a virtual game processing method provided in an embodiment of the present application when executed by a processor.

[0016] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions, which are used to implement the virtual game processing method provided in the embodiment of the present application when executed by a processor.

[0017] The embodiments of the present application have the following beneficial effects:

[0018] By collecting multi-directional monitoring images of each player participating in the virtual game (including a first image captured from the screen of the electronic device used by the player, and a second image captured from the player), it is convenient for the referee to watch and judge, which provides sufficient basis for the referee's judgment, thereby ensuring the fairness of the virtual game. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 1 is a schematic diagram of the architecture of a virtual game processing system 100 provided in an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of the structure of a terminal device 500 provided in an embodiment of the present application;

[0021] Figure 3 It is a flowchart of a method for processing a virtual game provided in an embodiment of the present application;

[0022] FIG. 4A to FIG. 4G It is a schematic diagram of an application scenario of a method for processing a virtual game provided in an embodiment of the present application;

[0023] Figure 5A and Figure 5B It is a flowchart of a method for processing a virtual game provided in an embodiment of the present application;

[0024] Figure 6 It is a schematic diagram of the architecture of a virtual game processing system provided in an embodiment of the present application;

[0025] Figure 7It is a flowchart of a method for processing a virtual game provided in an embodiment of the present application;

[0026] FIG. 8A to FIG. 8C It is a schematic diagram of an application scenario of the virtual game processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0028] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0029] It is understandable that in the embodiments of the present application, related data such as user information is involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0030] In the following description, the terms "first\second\..." are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\..." can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0032] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0033] 1) In response to: used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed may be executed.

[0034] 2) Virtual scene: It is the scene displayed (or provided) when the application is running on the terminal device. The virtual scene can be a simulation environment of the real world, a semi-simulation and semi-fictitious virtual environment, or a purely fictitious virtual environment. The virtual scene can be any one of a two-dimensional virtual scene, a 2.5-dimensional virtual scene, or a three-dimensional virtual scene. The embodiment of the present application does not limit the dimension of the virtual scene. For example, the virtual scene may include the sky, land, ocean, etc. The land may include environmental elements such as deserts and cities, and the player can control the player character to move in the virtual scene.

[0035] 3) Virtual game: refers to a game in which at least two virtual objects fight against each other in a virtual scene. The virtual game can be a single game in which at least two virtual objects fight against each other. The virtual game can include virtual objects from at least two virtual camps. Virtual objects from different camps win the virtual game by achieving designated battle goals. For example, the virtual objects of the first camp need to defend against the attack of the virtual objects of the second camp in a designated area within a limited time, while the virtual objects of the second camp need to attack the designated area within a limited time and occupy designated points in the designated area within the limited time.

[0036] 4) E-sports: It is a competition between people that combines intelligence and physical strength using electronic devices (such as terminal devices) as sports equipment.

[0037] 5) Real Time Messaging Protocol (RTMP): It is an open protocol developed by Adobe for the transmission of audio and video data between video players (such as Flash players) and servers. The RTMP protocol is like a container for data packets, which can be data in the Action Message Format (AMF) or audio and video data in the streaming media format (FLV, Flash Video).

[0038] 6) Secure Reliable Transport (SRT): It is a data transmission protocol (UDT) based on the User Datagram Protocol (UDP). It is safe, reliable and has low latency. SRT supports Advanced Encryption Standard (AES) encryption to ensure end-to-end video transmission security and forward error correction (FEC) to ensure transmission stability. Since SRT is built on the UDP protocol, it solves the problem of high transmission delay of the Transmission Control Protocol (TCP) and the complex transmission timing problem, and can support real-time transmission of high-throughput files and ultra-high-definition videos.

[0039] 7) Digital Video Recorder (DVR): A digital video stream recording service based on the Internet Protocol (IP), which regularly saves live video streams as disk files, such as MP4.

[0040] 8) Video Assistant Referee (VAR): Provides information to the referee on site through video playback, assists the referee in providing key video recording information, and improves the accuracy of penalty decisions.

[0041] The embodiments of the present application provide a method, device, electronic device, computer-readable storage medium and computer program product for processing a virtual game, which can provide a full range of information for the referee of the virtual game, facilitate the referee to watch and judge, thereby ensuring the fairness of the virtual game. The electronic device provided by the embodiment of the present application is described below. The electronic device provided by the embodiment of the present application can be implemented as a terminal device (such as a terminal device used by the referee in the virtual game), or it can be implemented in collaboration with a terminal device and a server. The following is an example of a method for processing a virtual game provided by an embodiment of the present application in collaboration with a terminal device and a server.

[0042] For example, see Figure 1 , Figure 1 1 is a schematic diagram of the architecture of a virtual game processing system 100 provided in an embodiment of the present application, in order to realize applications supporting and ensuring the fairness of virtual games, such as Figure 1As shown, the processing system 100 of the virtual game includes: a server 200, a network 300, terminal devices 400-1 to 400-N (where N is a positive integer greater than 1), and a terminal device 500, wherein the network 300 can be a local area network or a wide area network, or a combination of the two, and the terminal devices 400-1 to 400-N are terminal devices (i.e., player terminals) used by players 1 to N participating in the virtual game. Clients 410-1 to 410-N are respectively running on the terminal devices 400-1 to 400-N, and the clients 410-1 to 410-N are virtual scene clients of the same type, such as shooting game clients, multiplayer online tactical competitive game clients, role-playing game clients, real-time strategy game clients, racing game clients, music game clients, etc. Taking the terminal device 400-1 used by player 1 as an example, when player 1 participates in the virtual game, the terminal device 400-1 can capture the screen and call the built-in (or external) camera to collect images of player 1. Then the terminal device 400-1 can encode the collected video stream data and send the encoded video stream data to the server 200 through the network 300. In other words, the terminal device used by each player will capture the screen and collect the video stream data of the player to the server 200. The terminal device 500 is a terminal device used by the referee in the virtual game (i.e., the referee terminal). The client 510 runs on the terminal device 500. The client 510 is a client with the function of hosting an event, for example, it can be an e-sports assistant client, a social client based on competitive games, a live broadcast client, etc.

[0043] In some embodiments, a monitoring control may be displayed in the human-computer interaction interface of the client 510. For example, after the referee logs in to the client 510 using a pre-registered account (e.g., a referee code, used to represent the identity of the referee), a competition information aggregation page may be displayed in the human-computer interaction interface of the client 510, and a competition monitoring button may be displayed in the competition information aggregation page. Then, in response to the trigger operation on the monitoring control, the client 510 may display the screens corresponding to the multiple players included in at least one virtual match, wherein the types of screens include: a first screen (i.e., a screen recording screen) obtained by performing image acquisition on the screen of the terminal device used by the player, and a second screen (i.e., a player camera screen) obtained by performing image acquisition on the player. For example, when the client 510 receives a user's click operation on the competition monitoring button displayed in the competition information aggregation page, a request for obtaining video stream data may be sent to the server 200, and after receiving the video stream data returned by the server 200, the screens corresponding to the multiple players (e.g., player 1 to player N) included in at least one virtual match may be displayed.

[0044] It should be noted that an e-sports assistant APP can also be run on the terminal devices 400-1 to 400-N. For example, taking the terminal device 400-1 used by player 1 as an example, player 1 can use a pre-registered account (such as a player code, used to represent the player's identity) to log in to the e-sports assistant APP. When player 1 participates in the virtual game, the e-sports assistant APP can capture the screen of the terminal device 1 and call the built-in (or external) camera of the terminal device 1 to collect images of player 1. Subsequently, the e-sports assistant APP can send the collected video stream data to the server 200.

[0045] In other embodiments, the embodiments of the present application can also be implemented with the aid of cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to achieve data calculation, storage, processing, and sharing.

[0046] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool that can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources.

[0047] For example, Figure 1 The server 200 in the example may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms. The terminal devices 400-1 to 400-N, and the terminal device 500 may be smart phones, tablet computers, laptop computers, desktop computers, smart speakers, smart watches, vehicle-mounted terminals, etc., but are not limited thereto. The terminal devices (such as the terminal devices 400-1 to 400-N, and the terminal device 500) and the server 200 may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0048] In some embodiments, a terminal device (such as a terminal device used by a referee in a virtual game) can also implement the processing method of a virtual game provided in an embodiment of the present application by running various computer executable instructions or computer programs. For example, computer executable instructions can be microprogram-level commands, machine instructions, or software instructions. A computer program can be a native program or software module in an operating system; it can be a native application (APP, APPlication), that is, a program that needs to be installed in an operating system to run, such as an e-sports assistant APP or a browser; it can also be a small program, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module, or plug-in in any form.

[0049] The structure of the electronic device provided in the embodiment of the present application is further described below. Figure 1 Taking the terminal device 500 shown in FIG. 1 as an example, see Figure 2 , Figure 2 is a schematic diagram of the structure of a terminal device 500 provided in an embodiment of the present application, Figure 2 The terminal device 500 shown includes: at least one processor 520, a memory 560, at least one network interface 530 and a user interface 540. The various components in the terminal device 500 are coupled together via a bus system 550. It is understood that the bus system 550 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 550 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 550 is not shown in FIG. Figure 2 Various buses are labeled as bus system 550 .

[0050] The processor 520 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0051] The user interface 540 includes one or more output devices 541 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 540 also includes one or more input devices 542, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0052] The memory 560 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 560 may optionally include one or more storage devices that are physically remote from the processor 520.

[0053] The memory 560 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 560 described in the embodiments of the present application is intended to include any suitable type of memory.

[0054] In some embodiments, memory 560 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0055] Operating system 561, including system programs for processing various basic system services and performing hardware-related tasks, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0056] A network communication module 562, for reaching other computing devices via one or more (wired or wireless) network interfaces 530, exemplary network interfaces 530 include: Bluetooth, Wireless Compatibility Authentication (WiFi), and Universal Serial Bus (USB);

[0057] a presentation module 563 for enabling presentation of information via one or more output devices 541 (e.g., display screen, speaker, etc.) associated with the user interface 540 (e.g., a user interface for operating peripherals and displaying content and information);

[0058] The input processing module 564 is used to detect one or more user inputs or interactions from one of the one or more input devices 542 and translate the detected inputs or interactions.

[0059] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2The processing device 565 of the virtual game stored in the memory 560 is shown, which can be software in the form of a program or a plug-in, including the following software modules: a display module 5651, an exchange module 5652, an amplification module 5653, a playback module 5654, a receiving module 5655, a sending module 5656 and a determination module 5657. These modules are logical, and therefore can be arbitrarily combined or further split according to the functions implemented. It should be noted that in Figure 2 For the sake of convenience, all the above modules are shown at once, but it should not be considered that the processing device 565 of the virtual game excludes the implementation that can only include the display module 5651. The functions of each module will be explained below.

[0060] The following will specifically describe the method for processing the virtual game provided in the embodiment of the present application in combination with the exemplary application and implementation of the terminal device provided in the embodiment of the present application (for example, the terminal device used by the referee in the virtual game, i.e., the referee terminal).

[0061] See also Figure 3 , Figure 3 is a flowchart of a method for processing a virtual game provided in an embodiment of the present application, which will be combined with Figure 3 The steps shown are explained.

[0062] It should be noted that Figure 3 The method shown can be executed by various forms of computer programs running on the terminal device (i.e., the referee terminal), and is not limited to the client. For example, it can also be the operating system, software module, script, and applet mentioned above, so the example of the client in the following should not be regarded as a limitation on the embodiments of the present application. In addition, for the sake of convenience, the following does not specifically distinguish between the terminal device and the client running on the terminal device.

[0063] In step 101, a monitoring control is displayed in a human-computer interaction interface.

[0064] In some embodiments, an e-sports assistant APP can be run on the terminal device used by the referee of the virtual game. After the referee logs in to the e-sports assistant APP using a pre-registered account, the event information aggregation page can be displayed in the human-computer interaction interface of the e-sports assistant APP, and an event monitoring button (i.e., a monitoring control) can be displayed in the event information aggregation page.

[0065] In step 102, in response to a trigger operation on a monitoring control, screens corresponding to a plurality of players included in at least one virtual game are displayed.

[0066] Here, the types of pictures may include: a first picture obtained by capturing an image of a screen of an electronic device (i.e., a player end) used by a player (e.g., a picture of a virtual game from the player's first perspective, i.e., a recorded screen picture), and a second picture obtained by capturing an image of the player (e.g., a player's camera picture).

[0067] It should be noted that the first picture and the second picture in the embodiment of the present application can be pictures obtained by real-time acquisition, or can be pictures obtained in advance, such as pictures of pre-recorded videos, that is, the referee can replay the previously recorded video, and the embodiment of the present application does not make specific limitations on this.

[0068] In some embodiments, the above-mentioned display of the screens corresponding to the multiple players included in the at least one virtual game can be achieved in the following manner: for each of the multiple players included in the at least one virtual game, the following processing is performed: a first screen and a second screen corresponding to the player are displayed in a split-screen manner.

[0069] For example, taking at least one virtual game as virtual game A, assuming that virtual game A is a 5V5 match, that is, 10 players are required, assuming they are players 1 to 10 respectively, then when the referee's click operation on the monitoring control (such as the event monitoring button) is received, the screens corresponding to the 10 players participating in virtual game A (including the first screen and the second screen) can be displayed, wherein the first screen corresponding to each player (that is, the screen captured from the screen of the terminal device used by the player, such as the screen of the virtual game from the player's first perspective) and the second screen (that is, the screen obtained by capturing the image of the player) can be displayed independently in a split-screen manner.

[0070] In other embodiments, the above-mentioned display of the screens corresponding to the multiple players included in the at least one virtual game can also be achieved in the following way: for each of the multiple players included in the at least one virtual game, the following processing is performed: one of the first screen and the second screen corresponding to the player is displayed; in response to the screen switching operation, the other of the first screen and the second screen is switched to display.

[0071] For example, take at least one virtual game as virtual game A. Assume that virtual game A is a 5V5 match, that is, 10 players are required, and they are assumed to be players 1 to 10. Figure 4A , Figure 4A is a schematic diagram of an application scenario of the method for processing a virtual game provided in an embodiment of the present application, such as Figure 4AAs shown, a match monitoring button 402 is displayed in the match information aggregation page 401. When a click operation is received from the referee of the virtual game A on the match monitoring button 402, a first screen 403 (for example, a player's mobile phone screen stream, i.e., a screen captured from the screen of the mobile phone used by the player) corresponding to each of the 10 players participating in the virtual game A can be displayed in the match information aggregation page 401. Subsequently, when a click operation is received from the referee on the screen switching button 404 displayed in the match information aggregation page 401, a second screen 405 (for example, a camera stream, i.e., a screen obtained by capturing images of the players through a camera) corresponding to each of the 10 players participating in the virtual game A can be switched and displayed in the match information aggregation page 401.

[0072] It should be noted that in addition to uniformly switching the first screens corresponding to multiple players to the second screen, it is also possible to switch only the first screens corresponding to some players to the second screen. For example, when a click operation from the referee on the first screen 403 corresponding to player 1 is received, only the first screen 403 corresponding to player 1 can be switched to the second screen 405, and the other players remain unchanged, that is, for players 2 to 10, the corresponding first screen 403 is still displayed in the event information aggregation page 401. The embodiment of the present application does not make specific limitations on this.

[0073] In some embodiments, the above-mentioned display of the screens corresponding to the multiple players included in at least one virtual game can also be achieved in the following way: for each of the multiple players included in the at least one virtual game, the following processing is performed: one of the first screen and the second screen corresponding to the player adopts a floating window mode and is displayed in a floating manner in the other.

[0074] For example, taking at least one virtual match as virtual match A, assuming that virtual match A is a 5V5 match, that is, 10 players are required, assuming that they are players 1 to 10, then when a click operation of the referee on the monitoring control (such as the event monitoring button) is received, the screens (including the first screen and the second screen) corresponding to the 10 players participating in virtual match A can be displayed, wherein the first screen corresponding to each player can be displayed in the second screen in a floating window mode. In other words, the first screen and the second screen corresponding to each player can be displayed in a picture-in-picture manner.

[0075] It should be noted that the display modes of the screens corresponding to different players may also be different. For example, for player 1, the first screen corresponding to player 1 is displayed in a floating window mode in the second screen, that is, for player 1, the screen obtained by image acquisition of player 1 is mainly displayed; and for player 2, the second screen corresponding to player 2 is displayed in a floating window mode in the first screen, that is, for player 2, the screen captured from the screen of the terminal device used by player 2 (for example, the screen of the virtual game from the first perspective of player 2) may be mainly displayed. The embodiments of the present application do not make specific limitations on this.

[0076] In other embodiments, following the above example, the following processing may be performed: in response to a trigger operation on the first screen or the second screen, the display modes of the first screen and the second screen are exchanged. For example, assuming that the first screen is displayed in a floating window mode in the second screen, when a click operation on the first screen or the second screen is received by the referee, the first screen may be enlarged and the second screen may be reduced, so that the second screen is displayed in a floating window mode in the first screen.

[0077] For example, take at least one virtual game as virtual game A. Assume that virtual game A is a 5V5 match, that is, 10 players are required, and they are assumed to be players 1 to 10. Figure 4B , Figure 4B is a schematic diagram of an application scenario of the method for processing a virtual game provided in an embodiment of the present application, such as Figure 4B As shown, a match monitoring button 402 is displayed in the match information aggregation page 401. When a click operation on the match monitoring button 402 is received from the referee of the virtual game A, a first screen 403 and a second screen 405 corresponding to the 10 players participating in the virtual game A can be displayed in the match information aggregation page 401, wherein the second screen 405 is displayed in a floating window mode in the first screen 403. Taking player 1 as an example, when a click operation from the referee on the first screen 403 or the second screen 405 corresponding to player 1 is received, the display modes of the first screen 403 and the second screen 405 corresponding to player 1 can be exchanged. For example, the second screen 405 corresponding to player 1 is switched from the small window display mode to the normal display mode, while the first screen 403 is switched from the normal display mode to the small window display mode. That is to say, when a click operation from the referee on the first screen 403 or the second screen 405 corresponding to player 1 is received, the first screen 403 can be reduced and the second screen 405 can be enlarged, so that the first screen 403 is displayed in the floating window mode in the second screen 405, thereby facilitating the referee to view the content displayed on the second screen 405.

[0078] It should be noted that, in addition to exchanging the display modes of the first and second screens corresponding to a specific player, the display modes of the first and second screens corresponding to multiple players can also be exchanged in batches. For example, a screen mode switching button can also be displayed in the event information aggregation page. When a click operation of the referee on the screen mode switching button is received, the display modes of the first and second screens corresponding to multiple players can be exchanged in batches. Of course, when a click operation of the referee on the screen mode switching button is received, the referee can check the screens corresponding to multiple players, and then exchange the display modes of the first and second screens in the screens checked by the referee. This embodiment of the application does not specifically limit this.

[0079] In some embodiments, when displaying screens corresponding to multiple players included in at least one virtual game, the following processing can also be performed: displaying a player identifier (such as a player name, a player number, etc.) corresponding to each player; in response to a trigger operation on any player identifier, displaying a player details page of the player corresponding to the triggered player identifier.

[0080] For example, taking at least one virtual match as virtual match A, assuming that virtual match A is a 5V5 match, that is, 10 players are required, assuming that they are players 1 to 10, then while displaying the screens corresponding to the 10 players (including the first screen and the second screen), the player name corresponding to each player can also be displayed, for example, the player name of the player can be displayed below the screen corresponding to each player. When a click operation of the referee on any player name is received, the player details page of the player corresponding to the clicked player name can be displayed. For example, when a click operation of the referee on player name 2 is received, the player details page of the player corresponding to player name 2 (i.e., player 2) can be jumped from the event information aggregation page to the player details page of the player corresponding to player name 2.

[0081] In other embodiments, following the above example, the player details page may include at least one of the following information: the timeline of the virtual game in which the player participates, the performance parameters of the electronic device used by the player (i.e., the player end), such as the device IP, CPU temperature, memory usage, keyboard clicks per second (CPS, Click Per Second), mouse CPS, delay, etc., the first screen corresponding to the player, the second screen corresponding to the player, a marking control, and a download control, wherein the marking control is used to mark the player, for example, the player's avatar can be compared with the face image appearing in the second screen to prevent others from playing on his behalf; the download control is used to download a merged video, wherein the merged video refers to a video obtained by merging a first video stream corresponding to the first screen and a second video stream corresponding to the second screen.

[0082] For example, taking player 2 among multiple players as an example, at least one of the following information may be displayed in the player details page of player 2: a timeline of the virtual game in which player 2 participates; performance parameters sent by an electronic device used by player 2 (e.g., terminal device 2), such as the IP address of terminal device 2, CPU temperature, memory occupancy, number of keyboard clicks per second by player 2, number of mouse clicks per second by player 2, etc.; a first picture captured from the screen of terminal device 2 (i.e., the picture of the virtual game from the perspective of player 2); a second picture obtained by image acquisition of player 2; a player mark button for marking player 2; and a download button for downloading the merged video.

[0083] In some embodiments, the above-mentioned performance parameters, first screen and second screen can change dynamically with the progress of the virtual game. That is, the performance parameters, first screen and second screen displayed in the player details page correspond to the current progress of the virtual game. The referee can view the performance parameters corresponding to other progress by dragging the timeline of the virtual game. The following processing can also be performed: in response to the progress setting operation for the timeline of the virtual game, switch the display of the performance parameters, first screen and second screen corresponding to the set progress.

[0084] For example, taking player 2 among multiple players as an example, assuming that when the referee clicks on player name 2, the progress of the virtual game has just reached the 100th second, the performance parameters, the first picture, and the second picture corresponding to the 100th second can be displayed on the player details page of player 2, that is, the performance parameters sent by the player end of player 2 at the 100th second, the first picture captured on the screen of the player end of player 2, and the second picture obtained by image acquisition of player 2 can be displayed on the player details page of player 2. At the same time, the timeline of the virtual game can also be displayed on the player details page of player 2, so that the referee can adjust the progress of the virtual game. Assuming that the referee drags the progress of the virtual game forward by 10 seconds, the performance parameters, the first picture and the second picture corresponding to the 90th second can be updated and displayed in the player details page, that is, the performance parameters sent by the player end of player 2 at the 90th second, the first picture captured from the screen of the player end of player 2 at the 90th second, and the second picture obtained by image acquisition of player 2 at the 90th second are displayed in the player details page. In this way, the referee can backtrack, further ensuring the fairness of the virtual game.

[0085] In other embodiments, see Figure 5A , Figure 5A is a flowchart of a method for processing a virtual game provided in an embodiment of the present application, such as Figure 5A As shown, after executing Figure 3After step 102 shown, you can also perform Figure 5A Step 103A shown combines Figure 5A The steps shown are explained.

[0086] In step 103A, in response to a triggering operation on any screen, the triggered screen is enlarged, and at least one of the following controls is displayed: a marking control, a sharing control, and a download control.

[0087] In some embodiments, when a click operation of the referee on the screen corresponding to any player is received, the clicked screen can be enlarged, for example, it can be switched from displaying the screens corresponding to multiple players to only displaying the clicked screen, and at the same time, a mark control (such as a mark player button), a share control (such as a share video address button), and a download control (such as a download video button) can also be displayed below the clicked screen, wherein the mark control can be used to mark the player, for example, when the referee finds that the player may cheat through the enlarged screen, the player can be marked as a warning player through the mark control. In addition, when the referee is unsure whether the player cheats, the video can also be shared with other referees through the share control so that other referees can make a penalty together. In addition, the referee can also download the video through the download control, and when the subsequent player questions the penalty result, the downloaded video can be used as evidence.

[0088] It should be noted that in addition to capturing images of players through cameras, audio of players can also be captured through audio capture devices (such as microphones). For example, voice signals of players communicating with teammates during virtual games can be captured. When the triggered screen is amplified, the following processing can also be performed: play the audio signal obtained by audio capture of the player corresponding to the triggered screen, wherein the playback progress of the audio signal is aligned with the playback progress of the triggered screen.

[0089] For example, taking the triggered screen as the screen corresponding to player 1 as an example, while the screen corresponding to player 1 is amplified, for example, when switching from displaying screens corresponding to multiple players to displaying only the screen corresponding to player 1, the audio signal obtained by audio collection for player 1 can also be played, such as the voice signal when player 1 communicates with teammates by voice. In addition, in order to keep the audio and video synchronized, it can be achieved by aligning the timestamps of the picture and audio streams, for example, the audio stream can be aligned to the video stream, skipping frames if it is slow, and waiting if it is fast. That is to say, if the playback progress of the audio stream is slower than that of the video stream, the audio stream can be controlled to skip frames; if the playback progress of the audio stream is faster than that of the video stream, the audio stream can be paused first until the playback progress of the two is consistent, and then the audio stream can be continued to play, so as to ensure the smoothness of the video stream; of course, the video stream can also be aligned to the audio stream, and the frames can be skipped if it is slow, and waiting if it is fast, that is to say, if the playback progress of the video stream is slower than that of the audio stream, for example, assuming that the audio stream has been played to the 10th second, and the video stream has only been played to the 8th second, the video stream can be controlled to skip frames; if the playback progress of the video stream is faster than that of the audio stream, for example, assuming that when the video stream is played to the 10th second, the audio stream has only been played to the 9th second, the playback of the video stream can be paused first, until the playback progress of the two is consistent, and then the video stream can be continued to play, so as to ensure the smoothness of the audio stream.

[0090] In some embodiments, when displaying screens corresponding to multiple players included in at least one virtual game, the following processing can also be performed: displaying a player name input area; receiving an input player name through the player name input area; and updating the screen displaying the player corresponding to the input player name in response to a trigger operation on a query control.

[0091] For example, take at least one virtual game as virtual game A. Assume that virtual game A is a 5V5 match, that is, 10 players are required, and they are assumed to be players 1 to 10. Figure 4C , Figure 4C is a schematic diagram of an application scenario of the method for processing a virtual game provided in an embodiment of the present application, such as Figure 4CAs shown, in addition to displaying the first screen 403 and the second screen 405 corresponding to the 10 players respectively (wherein the first screen 403 and the second screen 405 can be independently displayed in a split screen manner), the event information aggregation page 401 can also display a player name input area 406. The referee can enter the name of the player to be searched in the player name input area 406. When the referee enters the name of the player to be searched (for example, the name of player 1) and receives the referee's click operation on the query button 407, the screen corresponding to player 1 (including the first screen 403 and the second screen 405) can be updated in the event information aggregation page 401. In this way, the referee's need to view specific players can be met.

[0092] In other embodiments, when displaying the screens corresponding to multiple players included in at least one virtual game, the following processing can also be performed: display a monitoring range selection control, wherein the types of monitoring range selection controls may include: viewing a complete monitoring control, viewing only a first-perspective control, and viewing only a camera monitoring control, and viewing the complete monitoring control is selected by default (i.e., the first screen and the second screen are displayed simultaneously by default); in response to a selection operation for viewing only the first-perspective control, updating and displaying the first screens corresponding to the multiple players; in response to a selection operation for viewing only the camera monitoring control, updating and displaying the second screens corresponding to the multiple players.

[0093] For example, take at least one virtual game as virtual game A. Assume that virtual game A is a 5V5 match, that is, 10 players are required, and they are assumed to be players 1 to 10. Figure 4D , Figure 4D is a schematic diagram of an application scenario of the method for processing a virtual game provided in an embodiment of the present application, such as Figure 4D As shown, the view complete monitoring control 408 is in the selected state by default, that is, by default, the first screen 403 and the second screen 405 corresponding to the 10 players are displayed simultaneously in the event information aggregation page 401. When the referee's selection operation for the view only first perspective control 409 is received, the first screen 403 corresponding to the 10 players can be updated and displayed in the event information aggregation page 401, that is, the second screen 405 corresponding to the 10 players is not displayed; when the referee's selection operation for the view only camera monitoring control 410 is received, the second screen 405 corresponding to the 10 players can be updated and displayed in the event information aggregation page 401, that is, the first screen 403 corresponding to the 10 players is not displayed. In this way, the referee's needs for viewing videos in different monitoring ranges can be met, and the user experience is improved.

[0094] In some embodiments, when displaying screens corresponding to multiple players included in at least one virtual game, the following processing can also be performed: displaying a match filtering control; in response to a trigger operation on the match filtering control, displaying multiple candidate matches; in response to a selection operation on multiple candidate matches, updating and displaying screens corresponding to multiple players included in the virtual game corresponding to the selected match, wherein at least one virtual game includes a virtual game corresponding to the selected match.

[0095] For example, take at least one virtual game as 3 virtual games, assuming they are virtual game A, virtual game B and virtual game C, where virtual game A includes players 1 to 4, virtual game B includes players 5 to 8, and virtual game C includes players 9 to 12, see Figure 4E , Figure 4E is a schematic diagram of an application scenario of the method for processing a virtual game provided in an embodiment of the present application, such as Figure 4E As shown, the event information aggregation page 401 displays the screens corresponding to the multiple players included in the three virtual games, including the first screen 403 and the second screen 405, wherein the first screen 403 and the second screen 405 are independently displayed in a split screen manner. The event information aggregation page 401 also displays a match filter control 411. When a click operation of the referee on the match filter control 411 is received, multiple candidate matches can be displayed, for example, including virtual match A, virtual match B, and virtual match C. When a click operation of the referee on virtual match B is received, that is, the referee selects the match corresponding to virtual match B from multiple candidate matches, then the screens corresponding to the multiple players included in virtual match B can be updated in the event information aggregation page 401, for example, including the screens corresponding to players 5 to 8, so that the referee can view the screens of multiple players included in the virtual match of a specific match.

[0096] In other embodiments, when displaying the screens corresponding to multiple players included in at least one virtual game, the following processing can also be performed: displaying a player range selection control, wherein the types of the player range selection control may include: an alarm player control, a non-alarm player control, and an unselected item control, and the unselected item control is in a selected state by default (i.e., the screens corresponding to all players are displayed by default); in response to a selection operation on the alarm player control, updating and displaying the screen corresponding to the alarmed player among the multiple players; in response to a selection operation on the non-alarm player control, updating and displaying the screen corresponding to the non-alarmed player among the multiple players.

[0097] For example, take at least one virtual game as virtual game A. Assume that virtual game A is a 5V5 game, that is, 10 players are required, and they are assumed to be players 1 to 10. Among them, players 1 to 4 are warning players (for example, players who may cheat), and players 5 to 10 are non-warning players. Figure 4F , Figure 4F is a schematic diagram of an application scenario of the method for processing a virtual game provided in an embodiment of the present application, such as Figure 4F As shown, the unselected item control 412 is in a selected state by default, that is, by default, the screens corresponding to the 10 players (including the first screen 403 and the second screen 405) are displayed simultaneously in the event information aggregation page 401. Subsequently, when the referee's selection operation for the warning player control 413 is received, the screen corresponding to the warning player can be updated in the event information aggregation page 401, for example, only the screens corresponding to players 1 to 4 are displayed in the event information aggregation page 401. When the referee's selection operation for the non-warning player control 414 is received, the screen corresponding to the non-warning player can be updated in the event information aggregation page 401, for example, only the screens corresponding to players 5 to 10 are displayed in the event information aggregation page 401, so that the referee's need to view the screens of players in a specific range can be met.

[0098] In some embodiments, before displaying the screens corresponding to the multiple players included in at least one virtual game, the following processing can also be performed: receiving video stream data respectively sent by multiple electronic devices (i.e., player terminals) corresponding to the multiple players one by one, wherein the video stream data is obtained by the electronic device capturing the screen of the electronic device and calling an image acquisition device (such as a built-in camera of the electronic device or an external camera) to capture images of the players.

[0099] For example, taking player 1 among multiple players as an example, assuming that the electronic device used by player 1 is terminal device 1, an e-sports assistant APP can be run on terminal device 1. When player 1 participates in the virtual game, the e-sports assistant APP can capture the screen of terminal device 1, that is, record the picture of the virtual game from the perspective of player 1. At the same time, the e-sports assistant APP can also call the built-in camera of terminal device 1 to capture images of player 1. Subsequently, the e-sports assistant APP can encode the two channels of video stream data obtained and send the encoded video stream data to the server.

[0100] In other embodiments, the video stream data may be encoded. In this case, before receiving the video stream data respectively sent by multiple electronic devices corresponding to multiple players, the following processing may be performed: encoding parameters adapted to the performance parameters of the electronic device are sent to each electronic device respectively, so that the electronic device encodes the video stream data based on the encoding parameters.

[0101] For example, in order to reduce the coding performance loss, coding parameters adapted to the player terminal can be issued according to the performance parameters of the player terminal (such as the device's computing speed, CPU main frequency, memory capacity, etc.), that is, different coding parameters are issued to different types of player terminals, so as to achieve a balance between coding quality, speed and device performance.

[0102] In some embodiments, see Figure 5B , Figure 5B is a flowchart of a method for processing a virtual game provided in an embodiment of the present application, such as Figure 5B As shown, after executing Figure 3 After step 102 shown, you can also perform Figure 5B Step 103B shown combines Figure 5B The steps shown are explained.

[0103] In step 103B, in response to a trigger operation on any screen, a plurality of images corresponding to the triggered screen are displayed.

[0104] Here, the multiple images may be obtained by taking multiple screenshots of the triggered screen.

[0105] For example, take at least one virtual game as virtual game A. Assume that virtual game A is a 5V5 match, that is, 10 players are required, and they are assumed to be players 1 to 10. Figure 4G , Figure 4G is a schematic diagram of an application scenario of the method for processing a virtual game provided in an embodiment of the present application, such as Figure 4G As shown, the screens corresponding to the 10 players (including the first screen 403 and the second screen 405) are displayed in the event information aggregation page 401. When a click operation of the referee on the screen corresponding to player 1 is received, multiple images 415 obtained by taking screenshots of the screen corresponding to player 1 (including the first screen and the second screen) can be displayed in the event information aggregation page 401. In this way, the referee can quickly review through the multiple images 415 without having to watch the video frame by frame, thereby improving the efficiency of penalty judgment.

[0106] It should be noted that the multiple images obtained by taking screenshots of the first screen and the multiple images obtained by taking screenshots of the second screen can be displayed separately, for example, the referee can switch them using a switch button; of course, the multiple images obtained by taking screenshots of the first screen and the second screen respectively can also be displayed together, and the embodiments of the present application do not specifically limit this.

[0107] In some embodiments, before displaying multiple images corresponding to the triggered screen, one of the following processes can also be performed: taking screenshots of the triggered screen at set time intervals (for example, 60 seconds) to obtain multiple images; calling a machine learning model to perform key frame prediction processing on the triggered screen to obtain multiple images, wherein the machine learning model is trained based on sample key frame images and label data marked for the sample key frame images.

[0108] For example, taking the triggered screen as the screen corresponding to player 1 as an example, the screen corresponding to player 1 can be screenshotted every 60 seconds to obtain multiple images; of course, the machine learning model can also be called to perform key frame prediction processing on the screen corresponding to player 1, that is, the machine learning model is used to determine whether the current frame is a key frame. If it is a key frame, a screenshot can be taken to obtain multiple images.

[0109] It should be noted that the above-mentioned machine learning model can be a neural network model (such as a convolutional neural network, a deep convolutional neural network, or a fully connected neural network, etc.), a decision tree model, a gradient boosting tree, a multi-layer perceptron, and a support vector machine, etc. The embodiment of the present application does not specifically limit the type of machine learning model.

[0110] The method for processing a virtual game provided in the embodiment of the present application collects multi-directional monitoring images of each player participating in the virtual game (including a first image captured by the screen of the electronic device used by the player, and a second image obtained by image acquisition of the player), so as to facilitate the referee to watch and judge, thereby ensuring the fairness of the virtual game.

[0111] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0112] The embodiment of the present application provides a method for processing a virtual game, constructs a set of auxiliary referee system based on audio and video, and has multi-faceted data monitoring and collection capabilities such as end-to-end screen recording collection, external camera and call voice collection, and end-to-end performance and network data collection. Through high-performance aggregation and processing such as recording, transcoding, and intelligent recognition marking in the audio and video background, it is finally presented to the referee user end (i.e., the referee end). It is an intuitive monitoring view that integrates multi-faceted monitoring video and intelligent marking of each player, as well as end-to-end device monitoring data, which is convenient for the referee to review and execute penalties.

[0113] The following is a detailed description of the method for processing a virtual game provided in an embodiment of the present application.

[0114] In some embodiments, see Figure 6 , Figure 6 is a schematic diagram of the architecture of a virtual game processing system provided in an embodiment of the present application, such as Figure 6 As shown, the architecture provided by the embodiment of the present application mainly includes: a data acquisition source and a data processing background, wherein the data acquisition source includes: external audio and video acquisition equipment, such as a point of view (POV) camera, an internal voice, a mobile phone camera, a microphone, etc.; the first-person audio and video acquisition and streaming in the event terminal, supporting Android, IOS and personal computers (PC, Personal Computer), etc.; the collection and reporting of device performance data, network data and behavior data in the competition terminal; the data processing background includes an audio and video background and a monitoring data background, wherein the audio and video background includes 5 functional modules, namely a live broadcast server, which is used to receive audio and video streaming data and convert the audio and video encapsulation format as needed; a video on demand server, which is used to receive video on demand requests from the client; a recording DVR, which is used to record the audio and video streams in the live broadcast server in real time and convert them into MP4 format files; a transcoding service, which is used to transcode the live stream in the live broadcast server, such as including mixed streams, resolution and bit rate conversion, etc.; a common gateway interface (CGI, Common The backend is used to manage and control the registration of user requests and backend servers; the monitoring data backend is used to receive the performance data and network quality detection data reported by the event client, and after data processing, it issues real-time alarms for abnormal data and provides a data interface for the front end to call.

[0115] Below Figure 6 The functions of each module shown in the figure are described in detail.

[0116] For example, the main functions of first-person screen recording and streaming are to collect the images in the game client and the sound of the microphone or speaker, and hard-code them through the APP program in the terminal to reduce the video size, and obtain the target streaming address through the link switching background, and then push the stream to the live broadcast server of the audio and video background.

[0117] For example, the main function of the external audio and video acquisition is to collect the POV camera image and intercom / microphone voice of the player's battle seat through the APP running on the PC, and then push the encoded stream to the live broadcast server of the audio and video background.

[0118] For example, the main function of the audio and video background is to record, store, transcode, and recognize the connected audio and video streams to achieve real-time monitoring, real-time playback, and screen marking. It is the core part of the video assisted referee system.

[0119] For example, the main function of the e-sports VAR unified monitoring station is to provide a web page (Web) for referees to operate and view monitoring data, which integrates the audio and video monitoring data of each player in the game and the quality monitoring data within the terminal.

[0120] In some embodiments, for the end-to-end audio and video acquisition, in order to reduce the coding performance loss, it can be achieved by utilizing hardware coding and encoder parameter optimization. For example, different coding parameters can be issued according to different models, so as to achieve a balance between coding quality, speed and device performance. In addition, for audio and video synchronization, the picture and audio stream (for example, which can be read through a speaker or microphone) can be aligned by calculating the timestamp. There are two ways. One is to align the audio stream to the video stream, skip frames if it is slow, and wait if it is fast, so as to ensure the smoothness of the video stream. The other is to align the video stream to the audio stream, and also skip frames if it is slow, and wait if it is fast, so as to ensure the smoothness of the audio stream.

[0121] In other embodiments, stream mixing and transcoding are mainly video stream mixing, which is convenient for video storage and front-end playback, saving traffic and storage space; screen recognition tags are mainly used to solve player head recognition and identity verification, to prevent cheating behaviors such as substitution and peeping at the big screen; high-performance storage and playback mainly rely on high-performance disk read and write rates. Mechanical disks (7200 rpm) will have limited read and write, while solid-state disks (SSD) will have higher read and write efficiency. Therefore, the recording and playback of multiple streams have certain requirements on hardware.

[0122] In some embodiments, see Figure 7 , Figure 7 is a flowchart of a method for processing a virtual game provided in an embodiment of the present application, which will be combined with Figure 7 The steps shown are explained.

[0123] In step 201, the collection end pushes the stream to the live streaming connection.

[0124] Here, the acquisition end refers to the screen recording and streaming in the terminal device (such as a mobile phone) used by the player, or other screen recording tools (such as VMIX or OBS, etc.), encoders and other encoding and streaming tools.

[0125] In step 202, the live streaming receiver sends a transcoding request to the transcoding and mixing unit.

[0126] Here, live streaming refers to a background streaming server that is used to receive video streams sent by a client (eg, a collection terminal).

[0127] In step 203, the transcoded mixed stream is transcoded.

[0128] Here, transcoding and mixing refers to the background service that can transcode the video stream, such as converting the format, converting the resolution, mixing the stream, etc., so as to generate a new video stream to adapt to the player and recording at the playback end. Here, mixing refers to mixing and splicing two or more streams into one stream to reduce storage space.

[0129] In step 204, the monitoring station sends a live streaming monitoring request to the transcoding and mixing station.

[0130] In step 205, the transcoded mixed stream is sent to the monitoring station as a live video stream.

[0131] In step 206, the transcoded mixed stream sends a recording request to the recording storage.

[0132] In step 207, the recording storage is recorded.

[0133] Here, recording and storage refers to recording the live stream or transcoded video stream into a file format such as MP4 or FLV for storage on a disk.

[0134] In step 208, the monitoring station sends a video-on-demand playback request to the video-on-demand service.

[0135] Here, the monitoring console refers to the entry page operated by the referee, which is used to watch live or on-demand video streams.

[0136] In step 209, the on-demand service reads the file from the recording storage.

[0137] Here, the main function of the on-demand service is to receive on-demand playback requests from clients or web pages.

[0138] In step 210, the on-demand service returns the video stream to the monitoring station.

[0139] In some embodiments, the technical solution provided by the embodiments of the present application can realize on-demand playback of real-time video recording of the game.

[0140] For example, see Fig. 8A , Fig. 8A is a schematic diagram of an application scenario of the method for processing a virtual game provided in an embodiment of the present application, such as Fig. 8A As shown, a competition monitoring button 802 is displayed in a monitoring aggregation page 801 (i.e., a competition information aggregation page). When a click operation is received from the referee on the competition monitoring button 802, pre-recorded video images 803 (including screen recording images and player POV camera images) corresponding to each player participating in the competition (i.e., a virtual game) can be displayed. When a click operation is received from the referee on the image 803 corresponding to any player, the clicked image 803 can be enlarged, and a plurality of buttons can be displayed below the clicked image 803, for example, a mark player button 804 for the referee to mark the player, a share video address button 805 for the referee to share the video address with others, and a download video button 806 for downloading the video.

[0141] In other embodiments, the technical solution provided in the embodiments of the present application can also realize real-time monitoring of game videos.

[0142] For example, see Figure 8B , Figure 8B is a schematic diagram of an application scenario of the method for processing a virtual game provided in an embodiment of the present application, such as Figure 8B As shown, a match monitoring button 802 is displayed in the monitoring aggregation page 801. When a click operation of the referee on the match monitoring button 802 is received, the screen recording screen 807 (corresponding to the first screen above) and the player POV camera screen 808 (corresponding to the second screen above) corresponding to each player participating in the game can be displayed. At the same time, the player's name 809 can also be displayed below the screen recording screen 807 corresponding to each player. When a click operation of the referee on the screen recording screen 807 corresponding to any player is received, the clicked screen recording screen 807 can be enlarged. When a click operation of the referee on the player name 809 corresponding to any player is received, the player details page 810 of the player (for example, player 6) corresponding to the clicked player name 809 can be displayed. Player monitoring details information can be displayed in the player details page 810, for example, including the IP address, CPU temperature, memory, keyboard CPS, mouse CPS, etc. of the terminal device used by player 6. At the same time, the player's corresponding screen recording screen, the player mark button, the download merged video button, etc. can also be displayed.

[0143] In other embodiments, see Figure 8C , Figure 8Cis a schematic diagram of an application scenario of the method for processing a virtual game provided in an embodiment of the present application, such as Figure 8C As shown, multiple images 811 obtained by real-time frame capture of the game operation screen can also be displayed in the monitoring aggregation page 801. For example, the game operation screen can be captured every 1 minute. Figure 1 times, thereby obtaining multiple images 811 for the referee to review.

[0144] To sum up, the method for processing virtual games provided in the embodiment of the present application is a panoramic data view formed by the real-time screen recording in the game client, the player's POV camera screen and call voice, the player's client performance and network quality data, etc., which provides a comprehensive, real-time, reliable and traceable basis for the execution referee to make anti-cheating judgments. Compared with the solutions provided by related technologies, it is difficult for the referee to obtain the above data for real-time traceback during the referee's execution. The technical solution provided in the embodiment of the present application can better ensure the fairness of the event.

[0145] The following is a description of an exemplary structure of a virtual game processing device 565 implemented as a software module provided in an embodiment of the present application. In some embodiments, Figure 2 As shown, the software modules stored in the virtual game processing device 565 of the memory 560 may include: a display module 5651.

[0146] Display module 5651 is used to display the monitoring control in the human-computer interaction interface; display module 5651 is also used to respond to the trigger operation of the monitoring control to display the screens corresponding to multiple players included in at least one virtual game, wherein the types of screens include: a first screen obtained by capturing an image of the screen of the electronic device used by the player, and a second screen obtained by capturing an image of the player.

[0147] In some embodiments, the display module 5651 is further configured to perform the following processing for each of the multiple players included in the at least one virtual game: displaying a first screen and a second screen corresponding to the player in a split-screen manner.

[0148] In some embodiments, the display module 5651 is further used to perform the following processing for each of the multiple players included in at least one virtual game: display one of the first screen and the second screen corresponding to the player; and switch to display the other of the first screen and the second screen in response to a screen switching operation.

[0149] In some embodiments, the display module 5651 is further used to perform the following processing for each of the multiple players included in at least one virtual game: one of the first screen and the second screen corresponding to the player adopts a floating window mode and is displayed in a floating manner in the other.

[0150] In some embodiments, the virtual game processing device 565 further includes an interaction module 5652 for exchanging the display modes of the first screen and the second screen in response to a trigger operation on the first screen or the second screen.

[0151] In some embodiments, the display module 5651 is further configured to display a player identification corresponding to each player; and to display a player details page of the player corresponding to the triggered player identification in response to a trigger operation on any player identification.

[0152] In some embodiments, the player details page includes at least one of the following: a timeline of virtual games in which the player participates, performance parameters of an electronic device used by the player, a first screen corresponding to the player, a second screen corresponding to the player, a mark control, and a download control.

[0153] In some embodiments, the performance parameters, the first screen, and the second screen change dynamically with the progress of the virtual game; the display module 5651 is also used to respond to the progress setting operation of the timeline of the virtual game, and switch the display of the performance parameters, the first screen, and the second screen corresponding to the set progress.

[0154] In some embodiments, the processing device 565 of the virtual game also includes a magnification module 5653, which is used to magnify the triggered screen in response to a trigger operation on any screen; the display module 5651 is also used to display at least one of the following controls: a mark control, a share control, and a download control.

[0155] In some embodiments, the virtual game processing device 565 further includes a playback module 5654 for playing an audio signal obtained by audio collection of a player corresponding to the triggered screen, wherein the playback progress of the audio signal is aligned with the playback progress of the triggered screen.

[0156] In some embodiments, the display module 5651 is also used to display the player name input area; the virtual game processing device 565 also includes a receiving module 5655, which is used to receive the input player name through the player name input area; the display module 5651 is also used to update the screen displaying the player corresponding to the input player name in response to the trigger operation of the query control.

[0157] In some embodiments, the display module 5651 is also used to display a monitoring range selection control, wherein the types of monitoring range selection controls include: viewing a complete monitoring control, viewing only a first-perspective control, and viewing only a camera monitoring control; viewing a complete monitoring control is selected by default; the display module 5651 is also used to update and display first screens corresponding to a plurality of players respectively in response to a selection operation for viewing only a first-perspective control; and to update and display second screens corresponding to a plurality of players respectively in response to a selection operation for viewing only a camera monitoring control.

[0158] In some embodiments, the display module 5651 is also used to display a match filtering control; and to display multiple candidate matches in response to a trigger operation on the match filtering control; in response to a selection operation on multiple candidate matches, update and display the screens corresponding to multiple players included in the virtual game corresponding to the selected match.

[0159] In some embodiments, the display module 5651 is also used to display a player range selection control, wherein the types of player range selection controls include: an alert player control, a non-alert player control, and an unselected item control, wherein the unselected item control is in a selected state by default; and is used to update and display a screen corresponding to an alerted player among multiple players in response to a selection operation on the alert player control; and to update and display a screen corresponding to a non-alert player among multiple players in response to a selection operation on the non-alert player control.

[0160] In some embodiments, the receiving module 5655 is further used to receive video stream data respectively sent by multiple electronic devices corresponding to multiple players, wherein the video stream data is obtained by the electronic device capturing the screen of the electronic device and calling the image acquisition device to capture images of the players.

[0161] In some embodiments, the video stream data is encoded; the virtual game processing device 565 also includes a sending module 5656, which is used to send encoding parameters adapted to the performance parameters of the electronic device to each electronic device, so that the electronic device encodes the video stream data based on the encoding parameters.

[0162] In some embodiments, the display module 5651 is further used to display multiple images corresponding to the triggered screen in response to a trigger operation on any screen, wherein the multiple images are obtained by taking screenshots of the triggered screen.

[0163] In some embodiments, the processing device 565 of the virtual game also includes a determination module 5657, which is used to perform one of the following processes: taking screenshots of the triggered screen at set time intervals to obtain multiple images; calling a machine learning model to perform key frame prediction processing on the triggered screen to obtain multiple images, wherein the machine learning model is trained based on sample key frame images and label data marked for the sample key frame images.

[0164] It should be noted that the description of the device in the embodiment of the present application is similar to the description of the method embodiment described above, and has similar beneficial effects as the method embodiment, so it will not be repeated. Figure 3 , Figure 5A ,or Figure 5B The present invention should be understood by referring to the description of any one of the accompanying drawings.

[0165] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the computer device executes the processing method of the virtual game described in the embodiment of the present application.

[0166] The embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the processor will execute the virtual game processing method provided by the embodiment of the present application, for example, Figure 3 , Figure 5A ,or Figure 5B The processing method of the virtual game is shown.

[0167] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0168] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0169] As an example, the executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0170] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for processing a virtual game, characterized in that: The method comprises: Display monitoring controls in the human-computer interaction interface; In response to a trigger operation on the monitoring control, screens corresponding to a plurality of players included in at least one virtual game are displayed, wherein the types of the screens include: a first screen obtained by capturing an image of a screen of an electronic device used by the player, and a second screen obtained by capturing an image of the player.

2. The method according to claim 1, characterized in that The displaying of the screens corresponding to the multiple players included in at least one virtual game includes: For each of the multiple players included in the at least one virtual game, the following process is performed: The first screen and the second screen corresponding to the player are displayed in a split-screen manner.

3. The method according to claim 1, characterized in that The displaying of the screens corresponding to the multiple players included in at least one virtual game includes: For each of the multiple players included in the at least one virtual game, the following process is performed: displaying one of the first screen and the second screen corresponding to the player; In response to a screen switching operation, the other of the first screen and the second screen is switched and displayed.

4. The method according to claim 1, characterized in that: The displaying of the screens corresponding to the multiple players included in at least one virtual game includes: For each of the multiple players included in the at least one virtual game, the following process is performed: One of the first screen and the second screen corresponding to the player adopts a floating window mode and is displayed in a floating manner in the other.

5. The method according to claim 4, characterized in that The method further comprises: In response to a trigger operation on the first screen or the second screen, display modes of the first screen and the second screen are exchanged.

6. The method according to any one of claims 1 to 3, characterized in that: When displaying screens corresponding to a plurality of players included in at least one virtual game, the method further includes: Displaying a player identification corresponding to each of the players; In response to a trigger operation on any of the player identifiers, a player details page of the player corresponding to the triggered player identifier is displayed.

7. The method according to claim 6, characterized in that The player details page includes at least one of the following: a timeline of the virtual game in which the player participates, performance parameters of the electronic device used by the player, the first screen corresponding to the player, the second screen corresponding to the player, a mark control, and a download control.

8. The method according to claim 7, characterized in that The performance parameter, the first screen, and the second screen dynamically change with the progress of the virtual game; The method further comprises: In response to a progress setting operation for the timeline of the virtual game, the performance parameters, the first screen, and the second screen corresponding to the set progress are switched for display.

9. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: In response to a triggering operation on any of the screens, the triggered screen is enlarged, and at least one of the following controls is displayed: a marking control, a sharing control, and a download control.

10. The method according to claim 9, characterized in that When the triggered picture is enlarged, the method further includes: An audio signal obtained by audio collection of a player corresponding to the triggered screen is played, wherein the playing progress of the audio signal is aligned with the playing progress of the triggered screen.

11. The method according to any one of claims 1 to 3, characterized in that: When displaying screens corresponding to a plurality of players included in at least one virtual game, the method further includes: Display the player name input area; receiving a player name inputted through the player name input area; In response to a trigger operation on the query control, a screen displaying a player corresponding to the input player name is updated.

12. The method according to any one of claims 1 to 3, characterized in that: When displaying screens corresponding to a plurality of players included in at least one virtual game, the method further includes: Display the monitoring range selection control, wherein the types of the monitoring range selection control include: view complete monitoring control, view only first-person perspective control, view only camera monitoring control; the view complete monitoring control is selected by default; In response to a selection operation on the first-view-only control, updating and displaying the first screens respectively corresponding to the plurality of players; In response to a selection operation on the camera-only monitoring control, the second screens corresponding to the plurality of players are updated and displayed.

13. The method according to any one of claims 1 to 3, characterized in that: When displaying screens corresponding to a plurality of players included in at least one virtual game, the method further includes: Display the session filter controls; In response to a triggering operation on the session filtering control, displaying a plurality of candidate sessions; In response to the selection operation for the plurality of candidate games, screens corresponding to the plurality of players included in the virtual game corresponding to the selected game are updated and displayed.

14. The method according to any one of claims 1 to 3, characterized in that: When displaying screens corresponding to a plurality of players included in at least one virtual game, the method further includes: Displaying a player range selection control, wherein the types of the player range selection control include: an alert player control, a non-alert player control, and an unselected item control, wherein the unselected item control is in a selected state by default; In response to a selection operation on the player alerting control, updating and displaying a screen corresponding to the player alerted among the multiple players; In response to a selection operation on the non-alerted player control, a screen corresponding to the non-alerted player among the multiple players is updated and displayed.

15. The method according to any one of claims 1 to 3, characterized in that: Before displaying the screens corresponding to the multiple players included in the at least one virtual game, the method further includes: Receive video stream data respectively sent by multiple electronic devices corresponding to the multiple players, wherein the video stream data is obtained by the electronic device capturing the screen of the electronic device and calling an image acquisition device to acquire images of the players.

16. The method according to claim 15, characterized in that The video stream data is encoded; Before receiving the video stream data respectively transmitted by a plurality of electronic devices corresponding to the plurality of players, the method further comprises: Encoding parameters adapted to the performance parameters of the electronic device are respectively sent to each of the electronic devices, so that the electronic device encodes the video stream data based on the encoding parameters.

17. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: In response to a triggering operation on any of the screens, a plurality of images corresponding to the triggered screen are displayed, wherein the plurality of images are obtained by taking screenshots of the triggered screen.

18. The method according to claim 17, characterized in that Before displaying the multiple images corresponding to the triggered pictures, the method further includes: Perform one of the following actions: Taking screenshots of the triggered screen at set intervals to obtain multiple images; A machine learning model is called to perform key frame prediction processing on the triggered picture to obtain multiple images, wherein the machine learning model is trained based on sample key frame images and label data marked for the sample key frame images.

19. A virtual game processing device, characterized in that: The device comprises: A display module, used to display monitoring controls in a human-computer interaction interface; The display module is further used to display screens corresponding to multiple players included in at least one virtual game in response to a trigger operation on the monitoring control, wherein the types of the screens include: a first screen obtained by capturing an image of a screen of an electronic device used by the player, and a second screen obtained by capturing an image of the player.

20. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the virtual game processing method described in any one of claims 1 to 18 when executing the executable instructions stored in the memory.

21. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a processor, the method for processing a virtual game as described in any one of claims 1 to 18 is implemented.

22. A computer program product comprising a computer program or computer executable instructions, characterized in that When the computer program or computer executable instructions are executed by a processor, the method for processing a virtual game as described in any one of claims 1 to 18 is implemented.