Information display method and device for virtual scene, equipment and storage medium

Abstract

The invention relates to an information display method and device for a virtual scene, equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: displaying a live broadcast picture of a virtual scene in a live broadcast interface; the virtual scene comprises one or more scene elements; receiving an element designation operation executed on the live broadcast picture; displaying related information of specified scene elements in the live broadcast interface; the specified scene element is a scene element displayed at an operation position of the element specified operation in the live broadcast picture. According to the scheme, the terminal can display the related information of the specified scene element based on the specified operation of the user on the element in the live broadcast picture, so that the display efficiency of the information for the virtual scene is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device and storage medium for displaying information in a virtual scene. Background Technology

[0002] With the rapid development of the Internet, live streaming has been applied to various industries. For example, live streaming is widely used in the e-sports industry.

[0003] In related technologies, esports live streaming clients can transmit live esports matches to viewers in different regions in real time, making it convenient for users to understand and follow esports events. When watching esports live streams, users may encounter situations where they are not familiar with the competitive objects or mechanics of the esports event. In such cases, users can search for relevant information on third-party websites.

[0004] However, the above solutions require users to manually search for unfamiliar content in esports events, which usually results in low accuracy and inefficient acquisition of relevant information for users. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for displaying information in virtual scenes, which can improve the display efficiency of information in virtual scenes during live streaming. The technical solution is as follows:

[0006] On the one hand, a method for displaying information in a virtual scene is provided, the method comprising:

[0007] The live stream interface displays a virtual scene; the virtual scene contains one or more scene elements.

[0008] Receive the element-specific operation to be performed on the live stream screen;

[0009] The live streaming interface displays relevant information about a specified scene element; the specified scene element is the scene element displayed at the operation position of the specified operation in the live streaming screen.

[0010] On the other hand, a method for displaying information in a virtual scene is provided, the method comprising:

[0011] The terminal device receives operation information specifying an element operation sent by the terminal device; the element-specified operation is received by the terminal device when displaying a live screen of a virtual scene in the live screen interface, and is an operation performed on the live screen; the virtual scene contains one or more scene elements;

[0012] Based on the operation information, relevant information of a specified scene element is obtained; the specified scene element is the scene element displayed at the operation position of the specified operation in the live broadcast screen.

[0013] The relevant information of the specified scene element is sent to the terminal device so that the terminal device can display the relevant information of the specified scene element in the live broadcast interface.

[0014] On the other hand, an information display device for a virtual scene is provided, the device comprising:

[0015] A live streaming display module is used to display the live streaming screen of a virtual scene in the live streaming interface; the virtual scene contains one or more scene elements;

[0016] An operation receiving module is used to receive element-specific operations performed on the live broadcast screen;

[0017] The information display module is used to display relevant information of a specified scene element in the live broadcast interface; the specified scene element is the scene element displayed at the operation position of the specified operation of the element in the live broadcast screen.

[0018] In one possible implementation, before displaying relevant information about a specified scene element in the live streaming interface, the device further includes:

[0019] The operation information sending module is used to send operation information for the specified operation of the element to the server;

[0020] The information receiving module is used to receive relevant information about the specified scene element returned by the server.

[0021] In one possible implementation, the operation information includes: the live broadcast screen and operation location information; the operation location information is used to indicate the operation location of the operation specified by the element.

[0022] In one possible implementation, the operation information includes: an operation area image; the operation area image is an image of the area at the operation location specified by the element in the live broadcast frame; or...

[0023] The operation information includes: an image of the operation area, and the element specifying the operation location; or,

[0024] The operation information includes: the timestamp corresponding to the live broadcast screen, and the operation position specified by the element.

[0025] In one possible implementation,

[0026] The operation receiving module is also used to receive query record display operations;

[0027] The information display module is also used to display operation record information corresponding to the scene element in the live broadcast interface; the operation record information is used to indicate the query popularity of the scene element.

[0028] In one possible implementation, the operation receiving module is configured to receive a specified type of operation performed on the live broadcast screen, as the element-specified operation, when the operation record information is displayed.

[0029] In one possible implementation,

[0030] The operation receiving module is also used to receive query record hiding operations;

[0031] The information display module is also used to cancel the display of the operation record information in the live broadcast interface.

[0032] In one possible implementation, when the specified scene element is a user-controlled virtual object within the virtual scene, the apparatus further includes:

[0033] The perspective switching module is used to switch the live broadcast perspective of the virtual scene to the perspective of the user corresponding to the specified scene element in response to receiving a perspective switching operation during the process of displaying relevant information of the specified scene element.

[0034] In one possible implementation, the viewpoint switching module is used for,

[0035] During the display of relevant information of the specified scene element, a perspective switching control is displayed; in response to receiving a trigger operation on the perspective switching control, the live broadcast perspective of the virtual scene is switched to the perspective of the user corresponding to the specified scene element.

[0036] In one possible implementation, the relevant information of the specified scene element includes one or more of the following:

[0037] The activity information of the specified scene element in the virtual scene;

[0038] The state information of the specified scene element in the virtual scene;

[0039] The description information of the specified scene element;

[0040] The teaching information related to the specified scene elements.

[0041] On the other hand, an information display device for a virtual scene is provided, the device comprising:

[0042] The operation information receiving module is used to receive operation information of element-specified operation sent by the terminal device; the element-specified operation is the operation performed on the live screen when the terminal device displays the live screen of the virtual scene in the live screen interface; the virtual scene contains one or more scene elements;

[0043] The information acquisition module is used to acquire relevant information of a specified scene element based on the operation information; the specified scene element is the scene element displayed at the operation position of the specified operation of the element in the live broadcast screen;

[0044] The information sending module is used to send relevant information of the specified scene element to the terminal device so that the terminal device can display the relevant information of the specified scene element in the live broadcast interface.

[0045] In one possible implementation, the information acquisition module is used for:

[0046] Obtain the operation area image indicated by the operation information; the operation area image is the area image of the operation position of the specified operation of the element in the live broadcast screen; input the operation area image into the element recognition model to obtain the identification information of the specified scene element output by the element recognition model; query the information corresponding to the identification information of the specified scene element as the relevant information of the specified scene element;

[0047] The element recognition model is a machine learning model trained using image samples and the annotation information; the annotation information is used to indicate the identification information of scene elements in the image samples.

[0048] In another aspect, a computer device is provided, the computer device comprising a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the information display method for virtual scenes as described above.

[0049] In another aspect, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the information display method for virtual scenes as described above.

[0050] In another aspect, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the information display method for a virtual scene provided in the various optional implementations described above.

[0051] The technical solution provided in this application may include the following beneficial effects:

[0052] In this embodiment, during the live streaming of a virtual scene, the live streaming interface of the terminal device can display the live stream. Users can perform element-specific operations on elements in the live stream. At this time, the live streaming interface will display relevant information about the scene element at the operation location of the element-specific operation. For example, if a user encounters a situation where they are not familiar with the current live stream content while watching the live stream, the user can perform the aforementioned element-specific operation on the scene element in the current live stream. The terminal device can quickly display the relevant information of the specified scene element in the live stream without requiring the user to manually perform a network search. This facilitates users in obtaining relevant information about e-sports events more conveniently while watching the live stream, thereby improving the display efficiency of information in the virtual scene during the live streaming of the virtual scene.

[0053] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0054] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0055] Figure 1 This is a system configuration diagram of an information display system for a virtual scene according to an embodiment of this application;

[0056] Figure 2 This is a flowchart of an information display method for a virtual scene provided in one embodiment of this application;

[0057] Figure 3 This is a flowchart of an information display method for a virtual scene provided in one embodiment of this application;

[0058] Figure 4 This is a flowchart of an information display method for a virtual scene provided in one embodiment of this application;

[0059] Figure 5 This is a flowchart of an information display method for a virtual scene provided in one embodiment of this application;

[0060] Figure 6 This is a schematic diagram illustrating an operation log information display provided in an embodiment of this application;

[0061] Figure 7 This is a flowchart of an information display method for a virtual scene provided in one embodiment of this application;

[0062] Figure 8 This is a schematic diagram of the view switching control in a virtual scene provided in an embodiment of this application;

[0063] Figure 9 This is a flowchart of an information display method for a virtual scene provided in one embodiment of this application;

[0064] Figure 10 This is a schematic diagram of the click behavior provided in the embodiments of this application;

[0065] Figure 11 This is a schematic diagram illustrating the hero-related content provided in the embodiments of this application;

[0066] Figure 12 This is a schematic diagram of a half-screen display of teaching content provided in an embodiment of this application;

[0067] Figure 13 This is a schematic diagram illustrating the full-screen display of teaching content provided in an embodiment of this application;

[0068] Figure 14 This is a block diagram of an information display device for a virtual scene provided in an exemplary embodiment of this application;

[0069] Figure 15 This is a block diagram of an information display device for a virtual scene provided in an exemplary embodiment of this application;

[0070] Figure 16 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0071] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0072] This application proposes an information display scheme for virtual scenes. This scheme can display relevant information about specified scene elements based on user-defined operations on elements in a live stream, thereby improving the efficiency of information display in virtual scenes. For ease of understanding, some concepts involved in this application are explained below.

[0073] 1) Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0074] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Among these, pre-trained models, also known as large-scale models or foundational models, can be widely applied to downstream tasks across various AI fields after fine-tuning. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0075] 2) Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning. Pre-trained models are the latest development in deep learning, integrating the above techniques.

[0076] 3) A virtual scene is a virtual scene displayed (or provided) by an application when it runs on a terminal. This virtual scene can be a simulation of a real-world environment, a semi-simulated / semi-fictional three-dimensional environment, or a purely fictional three-dimensional environment. A virtual scene can be any of a two-dimensional, 2.5-dimensional, or three-dimensional virtual scene. The following embodiments use a three-dimensional virtual scene as an example, but this is not a limitation. Optionally, the virtual scene can also be used for virtual scene battles between at least two virtual characters. Optionally, the virtual scene can also be used for battles between at least two virtual characters using virtual items. Optionally, the virtual scene can also be used for battles between at least two virtual characters using virtual items within a target area, where the target area shrinks over time within the virtual scene.

[0077] Virtual scenes are typically generated by applications on computer devices such as terminals and displayed based on the hardware (such as a screen) on those terminals. These terminals can be mobile devices such as smartphones, tablets, or e-readers; alternatively, they can be personal computer devices such as laptops or stationary computers.

[0078] The solutions provided in this application involve machine learning models for artificial intelligence, as illustrated in the following embodiments.

[0079] It should be noted that this application may display prompt interfaces, pop-ups, or output voice prompts before and during the collection of user-related data (such as user account information, operation information, etc.). These prompt interfaces, pop-ups, or voice prompts are used to inform the user that their relevant data is being collected. This ensures that the application only begins the steps related to collecting user-related data after receiving confirmation from the user regarding the prompt interface or pop-up; otherwise (i.e., without receiving confirmation from the user), the steps to collect user-related data end, meaning no user-related data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of relevant user data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0080] Figure 1 This is a system configuration diagram of an information display system for a virtual scene according to an embodiment of this application. Figure 1 As shown, the system includes a requesting terminal device 120 and a service device 130 for displaying information in a virtual scene.

[0081] The requesting terminal device 120 can be a personal computer device such as a desktop computer, laptop computer, tablet computer, mobile phone, or personal workstation.

[0082] The service provider equipment 130 can be a single server, or a combination of several servers, or a virtualization platform, or a cloud computing service center.

[0083] The requesting terminal device 120 and the service device 130 can be connected via a communication network. Optionally, this communication network can be a wired network or a wireless network.

[0084] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats, including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0085] exist Figure 1 In the system shown, the requesting terminal device 120 can request the service device 130 to provide relevant information of a specified scene element to be processed. The service device 130 sends the obtained relevant information of the specified scene element to the requesting terminal device 120, and the requesting terminal device 120 displays the relevant information of the specified scene element.

[0086] Figure 2 This is a flowchart illustrating an embodiment of an information display method for a virtual scene provided in this application. The information display method for a virtual scene can be executed by a computer device, for example, the computer device described above. Figure 1 The requesting terminal device 120 is shown. The above information display method may include the following steps:

[0087] Step 210: Display the live stream of the virtual scene in the live stream interface; the virtual scene contains one or more scene elements.

[0088] The aforementioned live streaming interface can be the interface or page displayed when a user watches a live stream on an online live streaming platform through a terminal device.

[0089] Optionally, the live streaming interface may include elements such as a video playback area (i.e., the current live content), a chat window, streamer information, and gift / reward buttons. Users can watch the streamer's live content on the live streaming interface and interact with other viewers or the streamer, such as sending messages, sending gifts, or giving likes through the chat window.

[0090] The aforementioned virtual scene can be a game-like virtual scene, and the game type can be, but is not limited to, any of the following:

[0091] 1) Shooting games: Players in this type of game can use virtual items to fight against players from the opposing faction. Examples include FPS (First-Person Shooter) games and TPS (Third-Person Shooter) games.

[0092] 2) MMORPG (Massively Multiplayer Online Role-Playing Game): This type of game allows hundreds or even thousands of players to play simultaneously in a virtual world. They can play the roles they create and engage in activities such as adventure, combat, and trading.

[0093] 3) MOBA (Multiplayer Online Battle Arena Games): MOBA is a popular esports game genre, representing multiplayer online tactical battle arena games. This game genre typically involves two opposing teams, each composed of multiple player-controlled characters. Players cooperate and use tactics to capture control points on the map or achieve specific objectives, ultimately defeating the opposing team; it's also a multiplayer shooter survival game.

[0094] 4) Sandbox Games: These games give players a great deal of freedom, allowing them to freely explore, create, and interact within the game.

[0095] 5) Strategy Games: Players in these games can develop long-term plans and have the ability to execute strategies and manage resources. Players typically face complex campaigns or economic systems and achieve game objectives through thinking, planning, and execution. Strategy games can be divided into various types, including Real-Time Strategy (RTS), Turn-Based Strategy (TBS), and Grand Strategy.

[0096] The aforementioned scene elements may include, but are not limited to:

[0097] 1) Virtual objects: such as virtual characters, virtual animals, etc.; among them, virtual objects can be divided into the first type of virtual objects controlled by players and the second type of virtual objects not controlled by players (such as virtual monsters controlled by AI), etc.; players can control the above-mentioned first type of virtual objects to move in the virtual scene (such as fighting).

[0098] 2) Virtual buildings: such as defense towers, attack towers, fountains, etc. deployed in virtual scenes. Virtual objects can move between virtual buildings.

[0099] 3) Virtual terrain: such as virtual hillsides, virtual rivers, virtual plants (such as grass / tree bushes), etc.

[0100] 4) Virtual props: such as virtual resources, virtual vehicles, etc. Virtual objects can use virtual props to achieve different effects and purposes in virtual scenes.

[0101] In some embodiments, the virtual scene includes one or at least two virtual objects. Players can control the virtual objects to move around in the virtual scene. The live streaming interface can display the scene of the virtual scene captured from the perspective of the player corresponding to the virtual object (that is, the scene of the virtual scene displayed on the terminal device of the player corresponding to the virtual object), or the scene of the virtual scene from the perspective of the streamer (that is, the scene of the virtual scene displayed on the terminal device of the streamer), or the scene of the virtual scene from the free perspective.

[0102] Step 220: Receive the element-specific operation to be performed on the live stream screen.

[0103] The aforementioned element-specific operation can be an operation in which a user selects one or at least two scene elements in the live stream through a terminal device.

[0104] For example, the operation forms specified by the above elements may include, but are not limited to: single click, double click, long press, swipe, etc.

[0105] In this embodiment of the application, the terminal device can receive element-specific operations performed by the user on one or at least two scene elements in the live broadcast.

[0106] In some embodiments, after receiving a user's specified operation on a scene element, the terminal device may highlight the scene element, zoom in / out on the scene element, highlight the scene element with text prompts, or play a voice prompt corresponding to the scene element (such as a voice announcement of the name of the scene element).

[0107] Step 230: Display relevant information of the specified scene element in the live broadcast interface; the specified scene element is the scene element displayed at the operation position of the specified operation in the live broadcast screen.

[0108] Among them, the above-mentioned relevant information is one or at least two pieces of information related to the scene element.

[0109] In some embodiments, the aforementioned relevant information may be collected in advance by the developers and stored in a database. When a user performs a specified operation on a scene element in the live streaming interface, the terminal device may query the database to obtain the relevant information of that scene element and display it.

[0110] In other embodiments, the aforementioned relevant information can also be automatically generated by a machine learning model in combination with the current scene image. In response to the user's specified operation on the current scene element, the terminal device can transmit the coordinates of the current scene element in the virtual scene and the current scene image to the server containing the machine learning model. After receiving the two types of data, the server inputs them into the machine learning model, which automatically generates the relevant information of the scene element and then sends the relevant information to the terminal device. The terminal device displays the aforementioned relevant information on the live streaming interface.

[0111] In some embodiments, the relevant information of a specified scene element may be displayed around the scene element (e.g., in a wraparound manner), or displayed in a new window created in the live streaming interface.

[0112] In this embodiment, during the live streaming of a virtual scene, the live streaming interface of the terminal device can display the live stream. Users can perform element-specific operations on elements in the live stream. At this time, the live streaming interface will display relevant information about the scene element at the operation location of the element-specific operation. For example, if a user encounters a situation where they are not familiar with the current live stream content while watching the live stream, the user can perform the aforementioned element-specific operation on the scene element in the current live stream. The terminal device can quickly display relevant information about the specified scene element in the live stream without requiring the user to manually perform a network search. This makes it easier for users to obtain relevant information in the virtual scene (such as the game scene of an e-sports event) while watching the live stream, thereby improving the display efficiency of information in the virtual scene during the live streaming of the virtual scene.

[0113] In some embodiments, information related to scene elements is specified, including one or more of the following:

[0114] Specify the activity information of scene elements in the virtual scene;

[0115] Specifies the state information of scene elements in the virtual scene;

[0116] Describe the descriptive information of the specified scene elements;

[0117] Teaching information related to specified scene elements.

[0118] The aforementioned activity information refers to the dynamic behavior or interaction of scene elements within the virtual scene. For example, this activity information may include actions that scene elements might perform, how they respond to player actions, and the progress or record of their activities within the virtual scene. For instance, if a scene element is a hostile, non-player-controlled virtual character, the activity information may include, but is not limited to, its patrol path and attack pattern.

[0119] In some embodiments, when the scene element is a virtual object controlled by a player or AI, the aforementioned activity information may include, but is not limited to: the number of times the virtual object defeats other virtual objects, the number of times the virtual object is defeated by other virtual objects, the number of times the virtual object assists other virtual objects, the amount of resources acquired, etc.

[0120] The aforementioned state information represents the characteristics of scene elements at a specific point in time within the virtual scene. This state information may include, but is not limited to:

[0121] 1) Physical state: such as whether scene elements are destroyed in the virtual scene, their movement status, their position and orientation, etc.

[0122] 2) Interaction status: When a scene element can interact with the player corresponding to that element, its status information may include, but is not limited to: whether it has been activated, whether it is in use, whether it is in a waiting state for interaction, etc.

[0123] 3) Functional status: For scene elements with specific functions (such as doors, machines, etc.), their status information may include one or more of the following: open / closed status, fault status, or running status.

[0124] In some embodiments, when the scene element is a virtual object controlled by a player or AI, the aforementioned state information may include, but is not limited to: attribute state (such as the current virtual object's health, mana, skill cooldown progress), equipment state (such as which virtual items are equipped, whether the equipped virtual items are effective, and whether the equipped virtual items are in use), and buff state (such as additional attribute bonuses given by teammates, additional attribute bonuses obtained by using virtual items, additional attribute bonuses obtained by defeating virtual monsters, etc.).

[0125] The aforementioned introductory information describes the basic information about the scene elements. This information can be presented to users in the virtual scene in the form of text, voice, or images to help them better understand and interact with the various scene elements. This information may include, but is not limited to:

[0126] 1) Name and appearance description: The name of the scene element and a description of its appearance, shape or features help users identify and understand the element.

[0127] 2) Functions and roles: Describe the functions and roles of scene elements in the virtual scene.

[0128] 3) Background story or history: The background story or history of the scene elements.

[0129] 4) Use tips or suggestions: Provide players with tips or suggestions that they may need when interacting with the scene elements so that they can better understand and utilize the elements.

[0130] In some embodiments, when the scene element is a virtual object, the above-described information may include, but is not limited to, the virtual object's name, background story, character type information, and usage information.

[0131] The aforementioned instructional information provides players with guidance or educational content on how to use, operate, or interact with specific scene elements. This information aims to help users understand the functions and roles of various scene elements within the virtual environment and how to interact with them effectively. The instructional information can guide users to understand the complexity and rules of the virtual environment and help them gradually improve their control skills and strategies regarding virtual elements. The aforementioned instructional information may include, but is not limited to:

[0132] 1) User Guide: Explains how users can control scene elements and interact with other elements in the virtual environment, such as which buttons, keyboard shortcuts, or gestures to use to perform specific actions.

[0133] 2) Hints and suggestions: Hints and suggestions on how to solve problems, complete tasks, or utilize scene elements.

[0134] 3) Tutorials and Demonstrations: Demonstrate how to use specific scene elements through tutorial levels, hints, or demonstrations of controlling scene elements in a virtual scene.

[0135] 4) Feedback and guidance: Provide real-time feedback as users interact with scene elements to help them understand how their actions affect the virtual scene and provide further guidance to achieve the desired results.

[0136] 5) Tasks or challenges: Design tasks or challenges that require users to learn and master specific skills or knowledge while interacting with scene elements.

[0137] In some embodiments, when the scene element is a virtual object, the above-mentioned teaching information may include, but is not limited to: recommended equipment for the virtual object, the order of moves for the virtual object, and different operation tutorials when the virtual object fights against different other virtual objects.

[0138] In this embodiment, the relevant information of scene elements is expanded to include multiple types of related information. The terminal device can display multiple types of related information on the live broadcast interface. Users can learn more about the live broadcast content through different related information, which helps users to better obtain relevant information of scene elements when watching the live broadcast, thereby improving the display efficiency of information in the virtual scene.

[0139] Please refer to Figure 3 This document illustrates a flowchart of an information display method for a virtual scene according to an embodiment of this application. For ease of explanation, the description focuses on a server or terminal as the executing entity for each step, wherein the server can... Figure 1 The service provider device 130 in the middle, the terminal device can be Figure 1 The requesting terminal device 120 in the method may include the following steps:

[0140] Step 310: The terminal device displays the live stream of the virtual scene in the live streaming interface; the virtual scene contains one or more scene elements.

[0141] Step 320: The terminal device receives the element-specific operation to be performed on the live broadcast screen.

[0142] Step 330: The terminal device sends the operation information for the specified operation to the server.

[0143] In this embodiment of the application, after receiving operation information from the user specifying an operation on an element in the live broadcast, the terminal device can send the operation information to the server through different network protocols. The sending methods may include, but are not limited to:

[0144] 1) HTTP (Hypertext Transfer Protocol) Request: Terminal devices can use HTTP to send operation information to servers over the network. Specifically, the terminal device constructs an HTTP request, which includes operation information as parameters or a request body, and then sends the request to a specific endpoint or URL (Uniform Resource Locator) on the server.

[0145] 2) Socket connection: Terminal devices can establish a persistent connection with the server through a socket and send operation information through this connection. This method is suitable for applications that require real-time communication or large amounts of data transmission.

[0146] 3) WebSocket: WebSocket is a protocol for full-duplex communication over a single TCP connection, typically used for real-time communication. Terminal devices can use WebSocket to establish a connection with a server and send operation information through that connection.

[0147] 4) UDP (User Datagram Protocol) packets: Terminal devices can use UDP to send packets to the server. This method is suitable for data transmission that requires fast speed and does not require acknowledgment.

[0148] Step 340: The server receives the operation information for the element specified by the terminal device.

[0149] The specified operation is the operation received by the terminal device when displaying the live screen of the virtual scene in the live screen interface, and is performed on the live screen; the virtual scene contains one or more scene elements.

[0150] In this embodiment, after the server receives the operation information sent by the terminal device, it parses and processes the received operation information according to the communication protocol. For example, if the operation information is sent via the HTTP protocol, the server will parse the HTTP request and extract the operation information; if the operation information is sent via a WebSocket or Socket connection, the server can parse the data packet according to the format specified by the WebSocket or Socket protocol and extract the operation information from the data packet.

[0151] Step 350: The server obtains relevant information about the specified scene element based on the operation information; the specified scene element is the scene element displayed at the operation position of the specified operation in the live broadcast.

[0152] In some embodiments, the server may include a database for storing relevant information about different scene elements. The server can query its own database based on the key information of the specified scene element indicated by the operation information, and find relevant information matching the operation information. For example, the server receives a specified operation information from a terminal device regarding hero A (the specified scene element) in a virtual scene. After parsing the specified operation information, the server obtains that hero A's name is "Z". The server then compares the name of each hero in the database storing relevant information about different heroes based on "Z". After finding the hero with the name "Z" in the database, the server retrieves the relevant information for that hero as the relevant information for the specified scene element.

[0153] In some embodiments, the server includes a machine learning model that can generate key information about specified scene elements based on operation information, and query relevant information about the corresponding specified scene elements using this key information. For example, after receiving operation information, the server sends the operation information to its internal machine learning model, which then automatically generates key information about the specified scene elements based on the operation information.

[0154] In some embodiments, after receiving operation information, the server can send the operation information to a third-party platform. This third-party platform includes a machine learning model, which can automatically generate key information for a specified scene element based on the operation information. For example, after receiving the operation information, the server uploads it to the third-party platform. The third-party platform then invokes the machine learning model to automatically generate key information for the specified scene element based on the received operation information and sends this key information to the server. The server then uses this key information to query relevant information about the corresponding specified scene element.

[0155] Step 360: The server sends the relevant information of the specified scene element to the terminal device so that the terminal device can display the relevant information of the specified scene element in the live broadcast interface.

[0156] In this embodiment of the application, the server can send relevant information about specified scene elements to the terminal device through different network protocols.

[0157] Step 370: The terminal device receives the relevant information of the specified scene element returned by the server.

[0158] Step 380: The terminal device displays relevant information about the specified scene element in the live broadcast interface; the specified scene element is the scene element displayed at the operation position of the specified operation in the live broadcast screen.

[0159] In this embodiment, the terminal device can send operation information of the specified operation to the server. Based on the operation information, the server can obtain relevant information of the specified scene element and send the relevant information to the terminal device. The terminal device can then display the information, which can quickly display the relevant information of the specified scene element in the live broadcast. This helps users to better obtain relevant information of scene elements in the virtual scene while watching the live broadcast, thereby improving the display efficiency of information in the virtual scene.

[0160] In some embodiments, the operation information includes: the live broadcast screen and operation location information; the operation location information is used to indicate the operation location of the operation specified by the element.

[0161] The aforementioned live stream footage can be automatically captured by the terminal device when a user performs a specified operation on an element of the live stream, and is the scene currently displayed on the live stream interface.

[0162] The aforementioned operation location information can be the coordinates of the specified operation in the scene interface / scene screen when a user watching the live stream performs the specified operation.

[0163] In this embodiment, the terminal device can directly obtain the live broadcast screen and operation location as the above-mentioned operation information based on the element-specified operation, thereby ensuring the efficiency of the terminal device in sending operation information to the server and reducing the complexity of the terminal device in obtaining operation information.

[0164] In some embodiments, the operation information includes: an operation area image; the operation area image is an image of the area at the operation location specified by the element in the live broadcast.

[0165] The aforementioned operation area image can be an image within a specified shape and size area in the live broadcast frame that contains the operation position specified by the aforementioned elements.

[0166] For example, taking a square image with a size of 256×256 pixels as an example, when the terminal device receives the operation specified by the element, it determines a square image area of ​​256×256 pixels with the operation position of the element specified as the center point. If the image area is completely within the live broadcast screen (that is, no point in the image area exceeds the boundary of the live broadcast screen), then the image within the image area is acquired as the operation area image. If the image area is not completely within the live broadcast screen (that is, some pixels in the image area exceed the live broadcast screen), then the position of the image area is adjusted so that the adjusted image area includes the operation position of the element specified as above and is completely within the live broadcast screen, and then the image of the adjusted image area is acquired as the operation area image. For example, if the right side of the image area extends beyond the live stream frame, the image area is moved to the left so that its right edge coincides with the right edge of the live stream frame; if the left side of the image area extends beyond the live stream frame, the image area is moved to the right so that its left edge coincides with the left edge of the live stream frame; if the top of the image area extends beyond the live stream frame, the image area is moved downwards so that its top edge coincides with the top edge of the live stream frame; if the bottom of the image area extends beyond the live stream frame, the image area is moved upwards so that its bottom edge coincides with the bottom edge of the live stream frame.

[0167] In some embodiments, the above operation information includes: an operation area image, and an element specifying the operation location.

[0168] In some embodiments, the above operation information includes: the timestamp corresponding to the live broadcast screen, and the operation position specified by the element.

[0169] The timestamp corresponding to the above live stream can be the playback time of the live stream.

[0170] Alternatively, the timestamp corresponding to the live stream can also be the time when the live stream was recorded. For example, when the recording segment of the live stream captures the live stream, it adds the recording time of each frame of the live stream as the timestamp of that frame to the live stream. When the terminal device receives the live stream and plays the live stream in the live stream interface according to the live stream, if it receives the operation specified by the above element, it extracts the timestamp corresponding to the currently playing live stream from the live stream.

[0171] In some embodiments, the server can obtain relevant information about a specified scene element based on the aforementioned operation area image. For example, after obtaining the operation area image, the server can analyze and process the operation area image (e.g., feature extraction, object detection, and classification) to determine the specified scene element at the location of the specified operation in the operation area image, and query relevant information about the specified scene element. Finally, the server can send the obtained relevant information about the specified scene element back to the terminal device for further processing or display.

[0172] In this embodiment, the operation information may not directly carry the complete live stream image, but may carry information indicating the operation area image in the live stream image. For example, the operation information may carry the operation area image, or the operation area image and operation location, or the timestamp and operation location, so that the server can determine the area image based on the operation information. Compared with operation information containing the complete live stream image, the operation information here has a smaller data volume, which can save traffic between terminal device servers, improve the efficiency of operation information reporting, and thus improve the display efficiency of information in the virtual scene.

[0173] based on Figure 3 Please refer to Figure 4 The diagram illustrates a flowchart of an information display method for a virtual scene provided in one embodiment of this application. Figure 3 Step 350 can be implemented as steps 350a, 350b, and 350c, with the server being the execution entity, as detailed below:

[0174] Step 350a: The server obtains the operation area image indicated by the operation information; the operation area image is the area image of the operation location specified by the element in the live broadcast screen.

[0175] For example, when the above operation information includes the live broadcast screen and the operation position of the element-specified operation, the server can determine the operation area image from the live broadcast screen based on the operation position of the element-specified operation. The process of the server determining the operation area image is similar to the process of the terminal device determining the operation area image, and will not be described in detail here.

[0176] For example, when the above operation information includes an operation area image, or when the above operation information includes an operation area image and the operation location specified by the element, the server directly extracts the operation area image carried in the operation information.

[0177] For example, when the above operation information includes the timestamp corresponding to the live stream and the operation position of the specified operation of the element, the server can query the live stream corresponding to the timestamp and determine the operation area image at the operation position of the specified operation of the element from the determined live stream. The process of the server determining the above operation area image is similar to the process of the terminal device determining the operation area image, and will not be described in detail here.

[0178] Step 350b: The server inputs the image of the operation area into the element recognition model and obtains the identification information of the specified scene elements output by the element recognition model.

[0179] The aforementioned element recognition model can be a machine learning model capable of outputting identification information of scene elements based on image information. The element recognition model can be a machine learning model built into the server, or it can be a third-party platform that includes an element recognition model.

[0180] In this embodiment, the element recognition model can be a machine learning model trained using image samples and labeling information samples; wherein, the aforementioned labeling information is used to indicate the labeling information of scene elements in the image samples. For example, during the training process of the element recognition model, the model training device can input image samples into the machine learning model to obtain the probability distribution of scene elements in the image samples belonging to various scene elements output by the machine learning model. The model training device calculates the loss function value based on the difference between the probability distribution and the labeling information, and updates the parameters of the machine learning model based on the loss function value.

[0181] The aforementioned identification information is used to identify scene elements. For example, the identification information can be the name or number of the scene element. A scene element can have one name or one number, and different scene elements have different names or numbers. The name of the scene element can be stored in text format, and the number of the scene element can be stored in numeric format.

[0182] In this embodiment of the application, after the server obtains the operation area image in the live broadcast, it passes the operation area image as input to the element recognition model. Then, the element recognition model uses a pre-trained neural network or other network to process the input image (e.g., feature extraction, classification, etc.) to identify the element in the image corresponding to the operation position of the specified operation of the above-mentioned element. Then, the element recognition model outputs the recognition result, such as the name or number of the above-mentioned scene element.

[0183] In another possible implementation, if the operation information includes the operation position of the specified operation, the server can also map the operation position of the specified operation (e.g., its coordinates in the live stream) to the coordinates of the corresponding operation position in the operation area image. Then, the operation area image and the coordinates of the specified operation position in the operation area image are input into the element recognition model to obtain the identification information of the specified scene element output by the element recognition model. Correspondingly, the element recognition model can be a machine learning model trained using image samples, the coordinates of scene elements in the image samples, and identification information samples. For example, during the training of the element recognition model, the model training device can input image samples and the coordinates of scene elements in the image samples into the machine learning model to obtain the probability distribution of the scene elements in the image samples belonging to various scene elements output by the machine learning model. The model training device calculates the loss function value based on the difference between this probability distribution and the annotation information, and updates the parameters of the machine learning model using this loss function value.

[0184] Step 350c: The server queries the information corresponding to the identifier information of the specified scene element, and uses it as the relevant information of the specified scene element.

[0185] In this embodiment of the application, the server may pre-store the identification information of each scene element and the correspondence between the relevant information of each scene element; after the server obtains the identification information of a specified scene element, it can query the information corresponding to the identification information of the specified scene element from the above correspondence and obtain the relevant information of the specified scene element.

[0186] In some embodiments, a scene element may correspond to multiple different related information. For example, a scene element may simultaneously correspond to activity information, status information, introduction information, and teaching information. Different types of related information may correspond to different data types (for example, the activity information and status information may be structured data, while the introduction information and teaching information may be unstructured data). Correspondingly, different data types may be suitable for different database structures. In this embodiment, at least two databases may be pre-set in the server. These at least two databases can be used to store different types of related information respectively. Correspondingly, after obtaining the identification information of a specified scene element, the server can query the information corresponding to the identification information of the specified scene element from the at least two databases respectively, and combine the information queried from the at least two databases to obtain the related information of the specified scene element.

[0187] For example, the server can contain two databases: a first database storing structured data and a second database storing unstructured data. The first database stores the identification information of scene elements and the correspondence between them and the scene elements' activity and status information. The second database stores the identification information of scene elements and the correspondence between them and the scene elements' description and teaching information. After obtaining the identification information of a specified scene element, the server can query the activity and status information of the specified scene element from the first database and the description and teaching information of the specified scene element from the second database. Then, the information retrieved from the two databases is merged to obtain the relevant information of the specified scene element.

[0188] In other embodiments, when the server fails to find relevant information about a specified scene element in the database based on the identification information, the server can call a web search engine to perform a web search. For example, the server can use the name of the specified scene element and the information type of the relevant information as search terms to obtain relevant information about the specified scene element. For example, if the information type of the relevant information is teaching information, the search term could be "tutorial on using xx hero (name of the specified scene element)". Afterward, the server will send the searched information to the terminal device.

[0189] In this embodiment, after the server obtains the image of the operation area indicated by the operation information, it obtains the identification information of the scene element according to the element recognition model, and then queries the database for relevant information corresponding to the identification information based on the identification information. This improves the accuracy of information query and thus improves the efficiency of information query, thereby improving the overall display efficiency of information for virtual scenes.

[0190] based on Figure 2 Please refer to Figure 5 , Figure 5 This is a flowchart of an information display method for a virtual scene provided in one embodiment of this application. The information display method may further include the following steps:

[0191] Step 240: The terminal device receives the query record display operation.

[0192] The query record display operation is used to instruct the terminal device to display the query records of each scene element in the virtual scene on the live broadcast interface.

[0193] In some embodiments, a display control may be displayed in the live streaming interface. When the user clicks the display control, the terminal device can determine that the above-mentioned query record display operation has been received.

[0194] In other embodiments, the user can trigger the terminal device to receive the above-mentioned query record display operation via voice command or gesture command. That is, when the terminal device receives the user's specified voice command or specified gesture command, it determines that it has received the above-mentioned query record display operation.

[0195] Step 250: On the live streaming interface, the terminal device displays operation record information for the corresponding scene element; the operation record information is used to indicate the query popularity of the scene element.

[0196] In some embodiments, the above-mentioned query popularity can be the number of times the corresponding scene element is subjected to element-specified operation during the live broadcast of the virtual scene, such as the total number of times the corresponding scene element has been subjected to element-specified operation during the time period from the start of the broadcast to the time when the terminal device displays the operation record information; or it can be the number of people who perform element-specified operation on the scene element during the live broadcast of the virtual scene, such as the total number of times the scene element is performed by the person during the time period from the start of the broadcast to the time when the terminal device displays the operation record information (it should be noted that when a person performs element-specified operation on the scene element multiple times, it is only counted as 1 time).

[0197] It should be noted that, in this embodiment of the application, the display of the above-mentioned query popularity is not limited to the time period from the start of the broadcast to the time when the terminal device displays the operation record information. After the terminal device starts to display the operation record information, when the user or the player corresponding to other virtual objects clicks on the scene element, the operation record of the scene element will be incremented by one.

[0198] Optionally, the above query popularity can be the query popularity of scene elements from the start of the live broadcast in the virtual scene to the current moment. For example, it can be the number of times the scene element received the specified operation from the start of the live broadcast in the virtual scene to the current moment, or the number of people who performed the specified operation on the scene element.

[0199] Alternatively, the above query popularity can also be the query popularity of the scene element within a specified time period before the current moment. For example, it could be the number of times the scene element received a specified operation within 3 minutes before the current moment, or the number of people who performed the specified operation on the scene element.

[0200] In some embodiments, when the terminal device displays operation record information in the live broadcast interface, different query popularity can be displayed on the corresponding scene elements with different effects. For example, the values ​​corresponding to the query popularity (such as the number of times the scene element received the specified operation, or the number of people who performed the specified operation on the scene element) are divided into three intervals in descending order. Scene elements with query popularity values ​​in the highest interval are displayed in red in the live broadcast interface, scene elements with query popularity values ​​in the middle interval are displayed in yellow in the live broadcast interface, and scene elements with query popularity values ​​in the lowest interval are displayed in green in the live broadcast interface. For example, in the current virtual scene, scene element A is queried 98 times, scene element B is queried 75 times, and scene element C is queried 50 times. The three intervals corresponding to the query counts, from high to low, are [+∞, 80], (80, 60], and (60, 0], respectively. In the live broadcast interface, scene element A is displayed with a red effect, scene element B with a yellow effect, and scene element C with a green effect.

[0201] For example, please refer to Figure 6 , Figure 6 This is a schematic diagram illustrating the display of operation log information provided in an embodiment of this application.

[0202] The live stream interface displays the live stream of virtual scene 600a, which contains four scene elements (virtual object 620a, virtual object 630a, virtual object 640a and virtual object 650a). The live stream interface also contains a display control 610a.

[0203] After a user watching the live stream clicks the display control 610a, the terminal device responds by recording and displaying the operation. In the live stream interface, the operation record information is displayed for the corresponding scene elements (virtual object 620b, virtual object 630b, virtual object 640b, and virtual object 650b). At this time, in the live stream screen of virtual scene 600b, virtual object 620b displays a query popularity of 36 (meaning that virtual object 620b has been clicked and queried by 36 people), virtual object 630b displays a query popularity of 15 (meaning that virtual object 630b has been clicked and queried by 15 people), virtual object 640b displays a query popularity of 58 (meaning that virtual object 640b has been clicked and queried by 58 people), and virtual object 650b displays a query popularity of 28 (meaning that virtual object 650b has been clicked and queried by 28 people).

[0204] In some embodiments, when the scene element is a virtual object and operation record information is displayed for the corresponding virtual object in the live broadcast interface, the operation record information is used to indicate the query popularity of the virtual object. A video display control for the corresponding virtual object is displayed around the operation record information. In response to the user's trigger operation on the video display control, the terminal device displays the playback video of the corresponding virtual object in the live broadcast interface. The playback video is a video that records the activities of the virtual object in the virtual scene.

[0205] Furthermore, when the search popularity of a virtual object reaches a specified threshold, the terminal device displays a replay video of the corresponding virtual object on the live streaming interface. The replay video is a video recording the activities of the virtual object in the virtual scene.

[0206] In this embodiment of the application, the aforementioned video collection can be generated by a terminal device or by a server.

[0207] In some embodiments, when a user clicks on a virtual object (i.e., a scene element controlled by the streamer / other user) in the live stream interface while watching a live stream, a control will be displayed around the clicked virtual object (which may be above or to the left of the control). After the user touches the control, a pop-up window will appear on the live stream interface. The pop-up window will automatically play a highlight reel of the virtual object within a specified time period (such as a season or a specific season). That is, it is a collection of combat videos of the same streamer / other user controlling the virtual object within a specified time period, and the combat videos in the collection of videos all meet specified conditions (e.g., meeting the requirement of killing a specified number of enemies within a specified time period).

[0208] Specifically, when a user clicks a control, the terminal device reads the user information (such as account information) of the virtual object corresponding to the control and sends the user information to the cloud server. After receiving the user information, the cloud server obtains at least two operation videos of the user controlling the virtual object within a specified time period, and then calls a machine learning model to automatically generate a set of videos based on the at least two operation videos; or the cloud server directly searches for the set of videos corresponding to the user from the cached set of videos database based on the user's account information. In the above-mentioned case, the cloud server can generate a set of videos database and automatically generate set of videos for each user at specified intervals and cache them in the set of videos database (for example, a set of videos corresponding to each user will be generated for each season).

[0209] The aforementioned video montage can be triggered when other users click on virtual objects in the virtual scene while watching the live stream; alternatively, while a user is watching the live stream, the terminal device can identify virtual objects in the current live stream interface, retrieve the video montage related to those virtual objects from the cloud server, and push it to the live stream interface. At this time, the user watching the live stream can click on the video montage to watch it in the window on the right side of the live stream interface, and the user can also play or close the window; the aforementioned video montage can also be automatically published on a platform (such as a live streaming platform) after being generated on the cloud server.

[0210] It should be noted that, in the above application embodiments, when displaying a collection video on the platform and displaying / publishing a collection video in the live broadcast interface, the terminal device will send a permission request prompt to the user corresponding to the virtual object in the collection video. If the user corresponding to the virtual object in the collection video allows permission, the collection video can play automatically.

[0211] In this embodiment of the application, users watching the live stream can trigger the terminal device to perform a recording and display operation, which helps users better understand the query popularity of scene elements in the current live stream, thereby more accurately triggering the terminal device to display the relevant information of the scene elements they want to obtain. This helps users better obtain relevant information of scene elements in the virtual scene while watching the live stream, thereby improving the display efficiency of information in the virtual scene.

[0212] In some embodiments, when displaying operation log information, the terminal device receives a specified type of operation performed on the live broadcast screen, which is used as an element to specify the operation.

[0213] The specified operation type mentioned above can be the operation of the user watching the live broadcast on scene elements in the live broadcast screen, including but not limited to: any one or at least two of the following: single click, double click, long press, voice command operation, and gesture command operation.

[0214] In this embodiment of the application, when operation record information appears in the live broadcast, the terminal device can detect the specified type of operation performed by the user to receive the specified operation of the element, which can avoid the user's misoperation when watching the live broadcast and ensure the information display quality of the virtual scene.

[0215] In some embodiments, the terminal device receives a query record hiding operation; and cancels the display of operation record information in the live broadcast interface.

[0216] In some embodiments, users watching a live stream can cancel the display of operation log information in the live stream interface by performing secondary operations on the display controls. For example, refer to... Figure 6 middle, Figure 6After the display control 610a receives the user's first trigger operation, it displays the record information of the battery life element in the scene interface 600b and changes to the state of the display control 610b. At this time, if the user triggers the display control 610b again, it is a second operation, and the operation record information in the scene interface is hidden.

[0217] In other embodiments, a timer can be set in the terminal device corresponding to the live broadcast screen. The timer has a preset duration set by the developer. The timer starts counting when the operation record information begins to be displayed. When the duration in the timer reaches the specified duration, the terminal device hides or cancels the display of the operation record information in the scene interface.

[0218] In other embodiments, users watching the live stream can also trigger the terminal device to cancel the display of operation record information through voice commands, gesture commands, or other means.

[0219] In this embodiment, the terminal device can respond to a user / server requesting a query record hiding operation, and cancel the display of operation record information in the live broadcast interface. This expands the method for displaying information in virtual scenes, making the display of information during the live broadcast more convenient and improving the efficiency of information display.

[0220] based on Figure 2 Please refer to Figure 7 , Figure 7 This is a flowchart of an information display method for a virtual scene according to an embodiment of this application. When the specified scene element is a virtual object controlled by the user in a virtual scene, the above information display method may further include the following steps:

[0221] Step 260: During the process of displaying relevant information of the specified scene element, the terminal device responds to the received view switching operation by switching the live broadcast view of the virtual scene to the view of the user corresponding to the specified scene element.

[0222] The aforementioned perspective switching operation can be performed by the user watching the live stream on the live stream interface (for example, triggering the display of the perspective switching control).

[0223] In this embodiment, the switched live stream screen is the scene screen displayed on the user's terminal corresponding to the specified scene element. This expands the live stream screen display method and the method for displaying information in virtual scenes, making the display of information during the live stream more convenient and improving the efficiency of information display.

[0224] In some embodiments, the terminal displays a view switching control while displaying relevant information of a specified scene element; in response to receiving a trigger operation on the view switching control, the live broadcast view of the virtual scene is switched to the view of the user corresponding to the specified scene element.

[0225] In this embodiment of the application, a perspective switching control is displayed on the terminal device. When a user watching the live stream triggers the perspective switching operation, the live stream interface switches the live stream perspective of the virtual scene to the perspective of the user corresponding to the specified scene element.

[0226] For example, please refer to Figure 8 , Figure 8 This is a schematic diagram of the perspective switching control in the virtual scene provided in this application embodiment. The live broadcast screen 800a includes a scene interface 810 and a scene interface 820. The scene interface 810 displays the game content currently being broadcast from a third-person perspective, while the scene interface 820 displays the view from the perspective of the player corresponding to a scene element in the scene interface 810. At this time, the user watching the live broadcast can trigger the perspective switching control 820a in the scene interface 820, and the live broadcast perspective in the live broadcast screen 800b will switch to the perspective of the user 801 corresponding to the specified scene element. The scene observed in the virtual scene from the perspective of the user 801 will be displayed in the scene interface 830.

[0227] In another possible implementation, the aforementioned perspective switching can also be performed automatically by the terminal device after certain conditions are met. For example, if the current live broadcast is the viewpoint of the broadcaster, after the terminal device obtains relevant information about a certain scene element and displays it for a certain period of time, if the scene element is a virtual object controlled by the user, the terminal device can switch to displaying the scene view from the user's perspective corresponding to that scene element.

[0228] In this embodiment of the application, users watching the live stream can select scene elements of interest and switch to the user's perspective corresponding to that element during the live stream, which expands the ways to switch perspectives during the live stream and improves the efficiency of users switching the perspective of the live stream.

[0229] based on Figures 2 to 5 ,as well as Figure 7 The solution, taking its application in MOBA games as an example, displays MOBA game scenes in the live stream. Please refer to [the provided text]. Figure 9 This document illustrates a flowchart of an information display method for a virtual scene according to an embodiment of this application. For ease of explanation, the description focuses on a server or terminal as the executing entity for each step. The method may include the following steps:

[0230] Step S1: The client records the player's click status.

[0231] The client records the player's current click status in real time. When a click is detected, it captures the live stream of the player's click and records the coordinates of the area the player clicked.

[0232] Step S2: AI analyzes the content clicked by the current player.

[0233] The recorded content is uploaded to an AI analysis platform. The platform automatically captures the player's clicked content based on the live stream footage and the coordinates of the clicked area, and then analyzes it using an AI model. The platform analyzes the clicked content and matches it with corresponding numbers.

[0234] Step S3: The server compares the player's current content with the server's tutorial content.

[0235] The clicked content label is synchronized to the server. The server then queries the database for the corresponding matching teaching content based on the label. If no match is found, the system returns a message to the client informing them not to proceed. If a match is found, the system returns the teaching content to the client.

[0236] Step S4: The client receives the teaching content returned by the server.

[0237] After receiving the tutorial content, the client determines the current player's state. If it's in a half-screen state, the tutorial content is displayed directly on the right. If it's in a full-screen state, the client exits full-screen mode and displays the tutorial content on the right.

[0238] Step S5: The client exits full-screen mode.

[0239] Players can view the tutorial content and click the 'x' to close the current tutorial page.

[0240] Based on the above process method, the specific application is as follows:

[0241] Part 1: Add click behavior recording to live stream footage

[0242] 1. The live streaming room has two states: half-screen state and full-screen state.

[0243] 2. Add user click behavior recording to the live stream, which is triggered when the user clicks the screen.

[0244] 3. After the user clicks the screen, the client records the screen content at the moment of the click and the area clicked by the user.

[0245] For example, please refer to Figure 10 , Figure 10 This is a schematic diagram of a click behavior provided in an embodiment of this application. In the live broadcast screen 1000, the user clicks on a virtual object 1001 in the virtual scene. At this time, the client records the screen content and the clicked area at the moment of the user's click.

[0246] Part 2: AI Image Recognition

[0247] 1. The client uploads the content of the screen at the moment the user clicks and the area the user clicks to the cloud AI.

[0248] 2. The AI ​​is trained using a large amount of live video footage and event-related data to automatically identify the content clicked by users.

[0249] 3. The AI ​​will return the identified content to the server.

[0250] Part 3: Creating a Teaching Content Resource Library

[0251] 1. Customize teaching content based on the game's rules and content.

[0252] 2. The teaching content mainly includes the following related content (map buildings, map creatures, map elements, in-game items, summoner skills).

[0253] 3. Create corresponding teaching materials for each content, such as the following is an introduction to hero-related content.

[0254] For example, please refer to Figure 11 , Figure 11 This is a schematic diagram illustrating the hero-related content provided in the embodiments of this application. Figure 11 The text introduces hero characteristics, hero skills, hero short guides, recommended sets, and recommended combinations.

[0255] Part 4: Interactive Presentation of Teaching Content

[0256] 1. The server matches the content from the teaching material library based on the content clicked by the user.

[0257] 2. If the current user is in a half-screen state, the tutorial content will pop up directly on the right, and the user can close it by clicking the 'x'.

[0258] For example, please refer to Figure 12 , Figure 12 This is a schematic diagram of a half-screen display of teaching content provided in an embodiment of this application.

[0259] When the live stream screen 1210 is in a half-screen state on the live stream interface, if the user clicks on the live stream screen 1210, the teaching content will pop up directly on the right side of the live stream screen 1210.

[0260] 3. If the current user is in full-screen mode, first exit full-screen mode and switch to half-screen mode, and at the same time, pop up the tutorial content on the right. The user can close it directly by clicking the 'x'.

[0261] For example, please refer to Figure 13, Figure 13 This is a schematic diagram of the full-screen display of teaching content provided in the embodiments of this application. When the live broadcast screen 1310 in the live broadcast interface is in full-screen mode, the user clicks on the live broadcast screen 1310. At this time, the live broadcast screen 1310 first exits the full-screen mode and switches to half-screen mode, and then the teaching content pops up on the right side of the live broadcast screen 1310.

[0262] Based on the above Figure 12 and Figure 13 The following is a detailed introduction:

[0263] During the live stream, users can click on the scene elements they want to learn about. At this time, the terminal device responds to the user's trigger operation on the live stream screen, determines whether the current live stream screen is in half-screen mode, and if it is in half-screen mode, the teaching content interface pops up directly on the right side of the live stream screen; if it is in full-screen mode, the current live stream screen exits full-screen mode, switches to half-screen mode, and then displays the teaching content interface.

[0264] For example, such as Figure 12 In the live stream, when a user clicks on a scene element (virtual hero) in the live stream screen 1210, the terminal device responds by receiving the user's click operation. The terminal device first obtains the scene screen corresponding to the current click operation and the relevant information of the scene element (virtual hero) (such as the hero's name and the path the hero is on in the current scene screen), and sends the relevant information to the server. Then, the server reads the database, finds the corresponding teaching content in the database based on the relevant information of the current virtual hero, and sends it to the terminal device. Finally, the terminal device displays a vertical teaching content interface on the right side of the live stream screen 1210.

[0265] For example, such as Figure 13 In the live stream, when a user clicks on a scene element (virtual hero) in the live stream screen 1310, the terminal device responds to the user's click operation. Since the live stream screen 1310 is currently in full-screen mode, it immediately exits the full-screen mode and switches to half-screen mode. A vertical teaching content interface is then displayed on the right side of the live stream screen 1310 in half-screen mode.

[0266] In the two embodiments described above, the terminal device first obtains the scene screen corresponding to the current user's click operation and the relevant information of the scene elements (virtual heroes) (such as the hero's name and the path the hero is taking in the current scene screen), sends the relevant information to the server, then the server reads the database, finds the corresponding teaching content in the database based on the relevant information of the current virtual hero, and sends it to the terminal device. Finally, the terminal device displays a vertical teaching content interface on the right side of the live broadcast screen.

[0267] The vertical tutorial interface displays detailed tutorial content (hero encyclopedia) corresponding to the clicked virtual hero. The hero encyclopedia contains multiple modules: hero characteristics, skill introduction, short guides, recommended sets, and recommended combinations.

[0268] The aforementioned Hero Characteristics module displays the interface effects of the virtual hero; the aforementioned Skill Introduction module displays the skill icons of the virtual hero and a detailed introduction to each skill (including skill characteristics, skill cooldown time, and skill operation guide); the aforementioned Short Guide module introduces the skill combination release methods of the virtual hero; the aforementioned Recommended Set module introduces different equipment sets for the virtual hero in the virtual scene; and the aforementioned Recommended Combination module introduces information on different teammate heroes that the virtual hero can be paired with in the virtual scene, as well as detailed combination coefficients.

[0269] In this embodiment of the application, when the above-mentioned teaching content (hero encyclopedia) is displayed, the teaching content preset by the developer is displayed by default. However, the content that each module in the teaching content can display is not limited to the default content. The terminal device can also display at least two alternative schemes in the interface. In some embodiments, the user can click on different modules in the teaching content, and each module will pop up an interface that displays at least two schemes. The user can choose a scheme that is different from the preset teaching content (such as popular recommended equipment).

[0270] In this application embodiment, the system automatically identifies the content that the user wants to know through image recognition, such as information about heroes, players, map elements, and creatures. It then aggregates match data, hero information, and tutorial content to generate a tutorial database, thus covering all the content the user wants to know and assisting in viewing the game. Users can click and watch anytime.

[0271] Please refer to Figure 14 The diagram illustrates a block diagram of an information display device for a virtual scene provided in an exemplary embodiment of this application. This information display device can be implemented as all or part of a computer device through hardware or a combination of hardware and software, to achieve the above-described... Figures 2 to 5 ,as well as Figure 7 All or part of the steps in the illustrated embodiments. For example... Figure 14 As shown, the information display device includes:

[0272] The live streaming display module 1401 is used to display the live streaming screen of a virtual scene in the live streaming interface; the virtual scene contains one or more scene elements;

[0273] The operation receiving module 1402 is used to receive the specified operation of the element to be performed on the live screen;

[0274] The information display module 1403 is used to display relevant information of a specified scene element in the live broadcast interface; the specified scene element is the scene element displayed at the operation position of the specified operation of the element in the live broadcast screen.

[0275] In one possible implementation, before displaying information about a specified scene element in the live streaming interface, the device further includes:

[0276] The operation information sending module is used to send operation information for the specified operation of an element to the server;

[0277] The information receiving module is used to receive relevant information about specified scene elements returned by the server.

[0278] In one possible implementation, the operation information includes: the live screen and operation location information; the operation location information is used to indicate the operation location of the specified operation of the element.

[0279] In one possible implementation, the operation information includes: an operation area image; the operation area image is the area image at the operation location specified by the element in the live broadcast; or...

[0280] Operation information includes: an image of the operation area, and an element specifying the operation location; or,

[0281] The operation information includes: the timestamp corresponding to the live stream screen, and the operation position specified by the element.

[0282] In one possible implementation,

[0283] The operation receiving module 1402 is also used to receive query record display operations;

[0284] The information display module 1403 is also used to display operation record information for corresponding scene elements in the live broadcast interface; the operation record information is used to indicate the query popularity of scene elements.

[0285] In one possible implementation, the operation receiving module 1402 is used to receive a specified type of operation performed on the live screen as an element specifying the operation, while displaying operation record information.

[0286] In one possible implementation,

[0287] The operation receiving module 1402 is also used to receive query record hiding operations;

[0288] The information display module 1403 is also used to cancel the display of operation record information in the live broadcast interface.

[0289] In one possible implementation, where the specified scene element is a user-controlled virtual object within a virtual scene, the device further includes:

[0290] The perspective switching module is used to switch the live broadcast perspective of the virtual scene to the perspective of the user corresponding to the specified scene element in response to the received perspective switching operation while displaying relevant information of the specified scene element.

[0291] In one possible implementation, the viewpoint switching module is used for...

[0292] Display a view switching control while showing relevant information about a specified scene element; in response to receiving a trigger operation on the view switching control, switch the live view of the virtual scene to the view of the user corresponding to the specified scene element.

[0293] In one possible implementation, relevant information about the scene elements is specified, including one or more of the following:

[0294] Specify the activity information of scene elements in the virtual scene;

[0295] Specifies the state information of scene elements in the virtual scene;

[0296] Describe the descriptive information of the specified scene elements;

[0297] Teaching information related to specified scene elements.

[0298] Please refer to Figure 15 The diagram illustrates a block diagram of an information display device for a virtual scene provided in an exemplary embodiment of this application. This information display device can be implemented as all or part of a computer device through hardware or a combination of hardware and software, to achieve the above-described... Figure 3 and Figure 4 All or part of the steps in the illustrated embodiments. For example... Figure 15 As shown, the information display device includes:

[0299] The operation information receiving module 1501 is used to receive operation information of element-specified operation sent by the terminal device; the element-specified operation is the operation performed on the live screen when the terminal device displays the live screen of the virtual scene in the live screen interface; the virtual scene contains one or more scene elements;

[0300] The information acquisition module 1502 is used to acquire relevant information of a specified scene element based on the operation information; the specified scene element is the scene element displayed at the operation position of the specified operation in the live broadcast screen.

[0301] The information sending module 1503 is used to send relevant information of a specified scene element to the terminal device so that the terminal device can display the relevant information of the specified scene element in the live broadcast interface.

[0302] In one possible implementation, the information acquisition module 1502 is used for,

[0303] Obtain the operation area image indicated by the operation information; the operation area image is the area image of the operation position of the element specified in the live broadcast; input the operation area image into the element recognition model to obtain the identification information of the specified scene element output by the element recognition model; query the information corresponding to the identification information of the specified scene element as the relevant information of the specified scene element;

[0304] Among them, the element recognition model is a machine learning model trained using image samples and annotation information; the annotation information is used to indicate the identification information of scene elements in the image samples.

[0305] Please refer to Figure 16 , Figure 16 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application. The computer device 1600 includes a Central Processing Unit (CPU) 1601, a system memory 1604 including Random Access Memory (RAM) 1602 and Read-Only Memory (ROM) 1603, and a system bus 1605 connecting the system memory 1604 and the CPU 1601. The computer device 1600 also includes a basic input / output system (I / O system) 1606 to facilitate information transfer between various devices within the computer, and a mass storage device 1607 for storing the operating system 1613, application programs 1614, and other program modules 1615.

[0306] The basic input / output system 1606 includes a display 1608 for displaying information and an input device 1609 for user input, such as a mouse or keyboard. Both the display 1608 and the input device 1609 are connected to the central processing unit 1601 via an input / output controller 1610 connected to the system bus 1605. The basic input / output system 1606 may also include the input / output controller 1610 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1610 also provides output to a display screen, printer, or other types of output devices.

[0307] The mass storage device 1607 is connected to the central processing unit 1601 via a mass storage controller (not shown) connected to the system bus 1605. The mass storage device 1607 and its associated computer-readable media provide non-volatile storage for the computer device 1600. That is, the mass storage device 1607 may include computer-readable media (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0308] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM (Random Access Memory), ROM (Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 1604 and mass storage device 1607 described above can be collectively referred to as memory.

[0309] Computer device 1600 can be connected to the Internet or other network devices via network interface unit 1611 connected to the system bus 1605.

[0310] The memory also includes one or more programs stored in the memory, and the central processing unit 1601 implements these programs by executing them. Figures 3 to 6 All or some of the steps in the method shown.

[0311] In an exemplary embodiment, a chip is also provided, the chip including programmable logic circuitry and / or program instructions, which, when the chip is run on a computer device, are used to implement all or part of the steps of the methods shown in the above embodiments of this application.

[0312] In an exemplary embodiment, a computer program product is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions to implement all or part of the steps of the methods shown in the above embodiments of this application.

[0313] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores a computer program that is loaded and executed by a processor to implement all or part of the steps of the methods shown in the above embodiments of this application.

[0314] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0315] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0316] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An information display method for a virtual scene, characterized by, The method comprises: displaying a live picture of a virtual scene in a live interface; the virtual scene comprises one or more scene elements; receiving an element designation operation performed on the live picture; displaying relevant information of a designated scene element in the live interface; the designated scene element is a scene element displayed at an operation position of the element designation operation in the live picture.

2. The method of claim 1, wherein, Before the displaying of the relevant information of the designated scene element in the live interface, the method further comprises: sending operation information of the element designation operation to a server; receiving, from the server, the relevant information of the designated scene element.

3. The method of claim 2, wherein, The operation information comprises the live picture and operation position information, which indicates the operation position of the element designation operation.

4. The method of claim 2, wherein the operation information comprises an operation area image; the operation area image is an area image at the operation position of the element designation operation in the live picture; or the operation information comprises an operation area image and the operation position of the element designation operation; or the operation information comprises a timestamp corresponding to the live picture and the operation position of the element designation operation.

5. The method of claim 1, wherein, The method further comprises: receiving a query record display operation; displaying operation record information corresponding to the scene element in the live interface; the operation record information indicates the query popularity of the scene element.

6. The method of claim 5, wherein, The receiving of the element designation operation performed on the live picture comprises: in a case where the operation record information is displayed, receiving a designated type operation performed on the live picture as the element designation operation.

7. The method of claim 5, wherein, The method further comprises: receiving a query record hiding operation; canceling the display of the operation record information in the live interface.

8. The method of claim 1, wherein, In a case where the designated scene element is a virtual object controlled by a user in the virtual scene, the method further comprises: in a process of displaying the relevant information of the designated scene element, in response to receiving a view angle switching operation, switching a live view angle of the virtual scene to a view angle of the user corresponding to the designated scene element.

9. The method of claim 8, wherein, The switching of the live view angle of the virtual scene to the view angle of the user corresponding to the designated scene element in the process of displaying the relevant information of the designated scene element in response to receiving the view angle switching operation comprises: displaying a view angle switching control in the process of displaying the relevant information of the designated scene element; in response to receiving a triggering operation on the view angle switching control, switching the live view angle of the virtual scene to the view angle of the user corresponding to the designated scene element.

10. The method according to any one of claims 1 to 9, characterized in that, The relevant information of the designated scene element comprises one or more of the following information: activity information of the designated scene element in the virtual scene; state information of the designated scene element in the virtual scene; introduction information of the designated scene element; teaching information related to the designated scene element.

11. An information display method for a virtual scene, characterized by, The method comprises: receive operation information of an element designation operation sent by a terminal device; the element designation operation is an operation received by the terminal device when displaying a live picture of a virtual scene in a live interface, and the operation is performed on the live picture; the virtual scene contains one or more scene elements; based on the operation information, obtain related information of a designated scene element; the designated scene element is a scene element displayed at an operation position of the element designation operation in the live picture; send the related information of the designated scene element to the terminal device, so that the terminal device displays the related information of the designated scene element in the live interface.

12. The method of claim 11, wherein, The operation information-based obtaining of the related information of the designated scene element comprises: obtain an operation region image indicated by the operation information; the operation region image is a region image at the operation position of the element designation operation in the live picture; input the operation region image into the element recognition model to obtain identification information of the designated scene element output by the element recognition model; query information corresponding to the identification information of the designated scene element as the related information of the designated scene element; The element recognition model is a machine learning model trained through image samples and the annotation information; the annotation information is used to indicate identification information of scene elements in the image samples.

13. An information display device for a virtual scene, characterized by The apparatus comprises: a live picture display module configured to display a live picture of a virtual scene in a live interface; the virtual scene contains one or more scene elements; an operation receiving module configured to receive an element designation operation performed on the live picture; an information display module configured to display related information of a designated scene element in the live interface; the designated scene element is a scene element displayed at an operation position of the element designation operation in the live picture.

14. An information display device for a virtual scene, characterized by The apparatus comprises: an operation information receiving module configured to receive operation information of an element designation operation sent by a terminal device; the element designation operation is an operation received by the terminal device when displaying a live picture of a virtual scene in a live interface, and the operation is performed on the live picture; the virtual scene contains one or more scene elements; an information obtaining module configured to obtain related information of a designated scene element based on the operation information; the designated scene element is a scene element displayed at an operation position of the element designation operation in the live picture; an information sending module configured to send the related information of the designated scene element to the terminal device, so that the terminal device displays the related information of the designated scene element in the live interface.

15. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores at least one computer instruction; the at least one computer instruction is loaded and executed by the processor to implement the information display method for a virtual scene according to any one of claims 1 to 12.

16. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one computer instruction, which is loaded and executed by the processor to implement the information display method for a virtual scene according to any one of claims 1 to 12.

17. A computer program product, characterised in that, The computer program product comprises computer instructions stored in a computer readable storage medium; the computer instructions are read and executed by the processor of the computer device to implement the information display method for a virtual scene according to any one of claims 1 to 12.

Images