Virtual scene data processing method and device, equipment, storage medium and program product

Abstract

The invention provides a data processing method and device for a virtual scene, equipment, a storage medium and a program product. The method comprises the steps that virtual scene data are obtained, and the virtual scene data comprise multiple types of sub-data; obtaining explanation delay duration of the virtual scene data and feature tags respectively corresponding to the multiple types of sub-data; screening at least one type of sub-data from the multiple types of sub-data based on the feature tag of each type of sub-data and the explanation delay duration to obtain to-be-explained data; and generating explanation content of the virtual scene based on the to-be-explanated data. Through the method and the device, the fluency degree of explaining the virtual scene can be improved.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and more particularly to a data processing method, apparatus, device, storage medium, and program product for virtual scenes. Background Technology

[0002] In related technologies, live streaming or video of video games is very common, and AI commentary in virtual game scenes is receiving increasing attention. High-quality AI commentary is an important component of a game's vitality. Commentary latency refers to the delay between the on-screen content and the commentary within the virtual scene. It is one of the most important indicators for evaluating commentary quality. When the game's frame rate is low, optimizing the processing time of a single frame can effectively solve the latency problem. However, with the increasing frame rates of mainstream games, latency issues are becoming more pronounced.

[0003] Currently, there is no good way to alleviate the problem of narration delay in virtual scenes. Summary of the Invention

[0004] This application provides a data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product for virtual scenes, which can improve the fluency of narration of virtual scenes.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a data processing method for a virtual scene, the method comprising:

[0007] Acquire virtual scene data, wherein the virtual scene data includes multiple types of sub-data;

[0008] Obtain the narration delay duration of the virtual scene data and the feature labels corresponding to the multiple types of sub-data;

[0009] Based on the feature labels of each type of sub-data and the interpretation delay duration, at least one type of sub-data is filtered out from the multiple types of sub-data to obtain the data to be interpreted;

[0010] The narration content for the virtual scene is generated based on the data to be narrated.

[0011] This application provides a data processing device for a virtual scene, including:

[0012] A data acquisition module is used to acquire virtual scene data, wherein the virtual scene data includes multiple types of sub-data;

[0013] The data acquisition module is also used to acquire the narration delay duration of the virtual scene data and the feature labels corresponding to the multiple types of sub-data respectively;

[0014] The data pruning module is used to filter out at least one type of sub-data from the multiple types of sub-data based on the feature labels of each type of sub-data and the interpretation delay duration, so as to obtain the data to be interpreted;

[0015] The narration module is used to generate narration content for the virtual scene based on the data to be narrated.

[0016] This application provides an electronic device, the electronic device comprising:

[0017] Memory is used to store executable instructions or computer programs.

[0018] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data processing method for the virtual scene provided in the embodiments of this application.

[0019] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed by a processor, implement the data processing method for a virtual scene provided in this application.

[0020] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the data processing method for a virtual scene provided in this application.

[0021] The embodiments of this application have the following beneficial effects:

[0022] Based on the narration delay duration and feature tags of each sub-data in the virtual scene, sub-data is filtered out. Compared to the full narration solution in related technologies, this simplifies the data to be narrated in the virtual scene, resulting in a shorter delay between the narration content generated based on the data to be narrated and the current virtual scene screen, improving the smoothness of the virtual scene narration and saving the computing resources consumed in generating the virtual scene narration content. Compared to the solution in related technologies that directly drops frames from the virtual scene data based on the narration delay duration, this method uses the feature tags of the sub-data and the narration delay duration as the criteria to determine the data to be narrated, avoiding the loss of important content in the narration and improving the user's experience of listening to the virtual scene narration. Attached Figure Description

[0023] Figure 1 This is a schematic diagram illustrating the application mode of the data processing method for virtual scenes provided in the embodiments of this application;

[0024] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0025] Figure 3A This is a first flowchart illustrating the data processing method for a virtual scene provided in an embodiment of this application;

[0026] Figure 3B This is a schematic diagram of the second process of the data processing method for virtual scenes provided in the embodiments of this application;

[0027] Figure 3C This is a schematic diagram of the third process of the data processing method for a virtual scene provided in the embodiments of this application;

[0028] Figure 3D This is a schematic diagram of the fourth process of the data processing method for a virtual scene provided in the embodiments of this application;

[0029] Figure 3E This is a schematic diagram of the fifth process of the data processing method for a virtual scene provided in the embodiments of this application;

[0030] Figure 3F This is a schematic diagram of the sixth process of the data processing method for virtual scenes provided in the embodiments of this application;

[0031] Figure 3G This is a schematic diagram of the seventh process of the data processing method for virtual scenes provided in the embodiments of this application;

[0032] Figure 4 This is the eighth flowchart of the data processing method for virtual scenes provided in the embodiments of this application;

[0033] Figure 5 This is a schematic diagram illustrating the principle of frame loss processing provided in the embodiments of this application;

[0034] Figure 6 This is a first structural schematic diagram of the decision tree model provided in the embodiments of this application;

[0035] Figure 7 This is a first structural schematic diagram of the computation tree model provided in the embodiments of this application;

[0036] Figure 8 This is a schematic diagram of the second structure of the decision tree model provided in the embodiments of this application;

[0037] Figure 9 This is a schematic diagram of the second structure of the computation tree model provided in the embodiments of this application.

[0038] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0041] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0042] It should be noted that the data collection and processing in this application should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0043] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0045] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0046] 1) Responding to: used to indicate the conditions or states on which the operation is performed depends. When the conditions or states on which it depends are met, one or more operations can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0047] 2) Virtual Scene: This refers to the scene displayed (or provided) by the application when it runs on the terminal device. The virtual scene can be a simulation of the real world, a semi-simulated / semi-fictional virtual environment, or a purely fictional virtual environment. The virtual scene can be any of a two-dimensional, 2.5-dimensional, or three-dimensional virtual scene; this application does not limit the dimension of the virtual scene. For example, a virtual scene may include the sky, land, ocean, etc., and the land may include environmental elements such as deserts and cities. Players can control their character to move within this virtual scene.

[0048] 3) Player-Controlled Character (PCC): This refers to the character controlled by the player in the game. The player character can be a virtual image in the virtual scene that represents the player, such as a virtual character, virtual animal, or anime character. The player character has its own shape and volume in the virtual scene and occupies a part of the space in the virtual scene.

[0049] 4) Non-Player Characters (NPCs): These are characters in a game that are not controlled by the player. NPCs are generally controlled by computer artificial intelligence and have their own behavioral patterns. They can be divided into story NPCs, combat NPCs, and service NPCs, and sometimes NPCs have multiple functions. Story NPCs and service NPCs are usually not attackable, or if they are, they are attackable but will not attack proactively. In addition, some NPCs drop items, provide game information to players, or trigger plot events.

[0050] 5) Real-time AI commentary for games: A system that provides real-time commentary and broadcasts of game content, including in-game event broadcasts, in-game analysis broadcasts, and broadcasts of relevant information outside the game.

[0051] 6) Game Features: Basic feature data within the game, such as: game features of competitive games include: player features (player health, resources, skill release actions, defeat actions, etc.), non-player character features (non-player character health, location coordinates, etc.), and global features (in-game time, number of surviving players, etc.).

[0052] 7) Narrative Events: Broadcastable events identified based on game features, such as: when the virtual object controlled by the player has low health, a low health event is triggered and broadcast. In this embodiment, the narration can take the form of voice broadcast.

[0053] 8) Caching Queue: A data structure that uses a queue to cache data. In this embodiment, the data cached in the caching queue is virtual scene data used for narration. Without blocking the game side, the virtual scene data at the head of the caching queue is read sequentially to generate narration content, ensuring data ordering.

[0054] 9) Frame rate: The frequency (rate) at which a bitmap image appears continuously on a display, in units of frames.

[0055] 10) Frames Per Second (FPS): The number of frames per second that a game runs. Similar to video, the higher the game's frame rate, the smoother the video playback on the screen.

[0056] 11) Commentary Delay Duration: The time difference between the occurrence of a commentary event in the virtual scene and the moment the commentary voice is broadcast. It is approximately equal to the waiting time of the game data in the cache queue. The waiting time can be used to quantify and evaluate the commentary delay. When the virtual scene data includes multiple sub-data, the commentary delay duration of the virtual scene data includes: the sub-commentary delay duration of multiple sub-data of the virtual scene data, with one sub-data corresponding to one sub-commentary delay duration.

[0057] 12) Pruning Algorithm: This is a technique in search algorithms to cut off useless parts of the search tree. A search tree refers to a complete decision tree model where some useless subtrees (decision paths) are pruned to allow the search process to reach the target state more quickly. Pruning involves using certain judgments to avoid unnecessary traversal processes. Figuratively speaking, it's about cutting off the "branches" formed by certain nodes in the search tree, hence the name pruning. In this embodiment, the pruning algorithm works by filtering out data representations that don't need to be interpreted.

[0058] 13) Decision Tree Model: This is a machine learning model primarily used for classification and regression tasks. Named for its tree-like structure, it represents different decision-making processes in a hierarchical manner. Each node in the decision tree represents a feature or attribute in the dataset, while each branch of the tree represents a different value for that feature.

[0059] 14) Computational tree model: This is a statistical model primarily used to represent a decision rule, which includes all possible output results. In this embodiment, the computational tree model is used as a structure to store the duration intervals corresponding to the sub-data of the virtual scene, and redundant data is filtered out by traversing the nodes of the computational tree model.

[0060] In related technologies, the latency between AI-generated narration and the virtual scene's visuals is mitigated by dropping frames from the virtual scene data. However, this frame dropping can result in some important content in the virtual scene not being narrated, affecting the user's experience. Even when the narration latency is low, narrating the entire virtual scene data leads to redundant narration content, wasting the computational resources needed to generate it and consuming network resources, thus impacting the fluency of the narration.

[0061] This application provides a data processing method for virtual scenes, a data processing device for virtual scenes, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the fluency of narrating virtual scenes.

[0062] The following describes exemplary applications of the electronic devices provided in the embodiments of this application. These electronic devices can be implemented as terminal devices, such as laptops, tablets, desktop computers, set-top boxes, smart TVs, in-vehicle terminals, virtual reality (VR) devices, augmented reality (AR) devices, and other various types of terminals. They can also be implemented as servers. The following will describe exemplary applications when the electronic device is implemented as a terminal device or a server.

[0063] refer to Figure 1 , Figure 1 This is a schematic diagram illustrating the application mode of the data processing method for virtual scenes provided in the embodiments of this application; for example, Figure 1 The system involves server 200, network 300, terminal device 400, and database 500. The human-computer interaction interface 100 serves as the interface for terminal device 400, displaying the virtual scene. Terminal device 400 connects to server 200 via network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0064] For example, in game application scenarios, solutions for collaborative implementation between terminal devices and servers mainly involve two game modes: local game mode and cloud game mode. In local game mode, the terminal device and server collaboratively run the game processing logic. The player's input commands on the terminal device are partly processed by the terminal device's game logic, and partly by the server. Furthermore, the game logic processed by the server is often more complex and requires more computing power. In cloud game mode, the game logic is entirely processed by the server, and the cloud server renders the game scene data into audio and video streams, which are then transmitted to the terminal device for display over the network. The terminal device only needs basic streaming media playback capabilities and the ability to receive player commands and send them to the server.

[0065] The virtual scene data processing method provided in this application embodiment is applied to terminal device 400 and server 200, and is suitable for application modes that rely on the computing power of server 200 to complete virtual scene calculation and output virtual scene on terminal device 400.

[0066] Taking the visual perception of forming a virtual scene as an example, server 200 calculates display data related to the virtual scene (such as scene data) and sends it to terminal device 400 via network 300. Terminal device 400 relies on graphics computing hardware to load, parse, and render the calculated display data, and relies on graphics output hardware to output the virtual scene to form visual perception. For example, it can display two-dimensional video frames on the display screen of a smartphone, or project video frames to achieve a three-dimensional display effect on the lenses of augmented reality / virtual reality glasses. As for the perception of the form of the virtual scene, it can be understood that it can be achieved with the help of the corresponding hardware output of terminal device 400, such as using a microphone to form auditory perception, using a vibrator to form tactile perception, and so on.

[0067] As an example, the terminal device 400 runs a client (such as a network-based game application). During the operation of the client, the output includes a virtual scene for role-playing. The virtual scene can be an environment for game characters to interact with, such as a plain, street, valley, etc., for game characters to fight. The first virtual object can be a game character controlled by the user. That is, the first virtual object is controlled by the real user and will move in the virtual scene in response to the real user's operation on the controller (such as a touch screen, voice switch, keyboard, mouse, and joystick). For example, when the real user moves the joystick to the right, the first virtual object will move to the right in the virtual scene. It can also stay still, jump, and be controlled to shoot.

[0068] Taking a computer program as an example, in actual implementation, the terminal device 400 has an application that supports virtual scenes installed and running. This application can be any of the following: a first-person shooter (FPS) game, a third-person shooter game, a virtual reality application, a 3D map application, or a multiplayer survival game. Users use the terminal device 400 to manipulate virtual objects located in the virtual scene, and these activities include, but are not limited to: adjusting body posture, crawling, walking, running, riding, jumping, driving, picking up items, shooting, attacking, throwing, and constructing virtual buildings—at least one of these. Illustratively, the virtual object can be a virtual character, such as a realistic or anime character.

[0069] Example, Figure 1 The virtual scene displayed in the human-computer interaction interface 100 can be a game virtual scene, the server 200 can be a live streaming platform or a game platform server, the user can be a viewer watching a game live stream, and the database 500 is the database of the game platform or the live streaming platform. The server 200 sends the virtual scene data to the terminal device 400 so that the human-computer interaction interface 100 of the terminal device 400 displays the virtual scene. The following explanation uses the above example.

[0070] In some embodiments, if a user believes that the video or live stream of the virtual scene they are currently watching lacks narration, the terminal device 400 can be triggered to send a narration request to the server 200 via the network 300. After receiving the narration request, the server 200 calls the virtual scene data processing method of this application embodiment to generate narration content and sends the narration content to the terminal device 400. The terminal device 400 plays the audio of the narration content while displaying the screen of the virtual scene, and can also display the text of the narration content.

[0071] In some embodiments, the virtual scene data processing method of this application embodiment can also be applied to the following application scenarios: 1. Video narration, for example: in video platforms, some videos lack subtitles and background audio. If users lack narration content while watching the video, they can call the virtual scene data processing method of this application embodiment to generate narration content for the video watched by the user. 2. Live content narration, for example: some live streams lack narration content, such as live streams of handcrafting. If users lack narration content while watching the live stream, they can call the virtual scene data processing method of this application embodiment to generate narration content for the live stream content watched by the user.

[0072] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0073] This application embodiment can be implemented using database technology. A database, simply put, can be viewed as an electronic filing cabinet storing electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, capable of being shared by multiple users, having minimal redundancy, and being independent of application programs.

[0074] A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters or mobile devices; or according to the query language used, such as Structured Query Language (SQL) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, simultaneously supporting multiple query languages.

[0075] This application embodiment can also be implemented using cloud technology. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on cloud computing business models. It can form a resource pool, available on demand, offering flexibility and convenience. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, and driven by demands for search services, social networks, mobile commerce, and open collaboration, every item may eventually possess its own hash-coded identification mark, requiring transmission to a backend system for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0076] In some embodiments, the terminal device or server can implement the data processing method for the virtual scene provided in this application by running a computer program. For example, the computer executable instructions can be microprogram-level commands, machine instructions, or software instructions. The computer program can be a native program or software module in the operating system; it can be a native application (APP), that is, a program that needs to be installed in the operating system to run, such as a live streaming APP or an instant messaging APP; or it can be a mini-program that can be embedded in any APP, that is, a program that only needs to be downloaded to the browser environment to run. In summary, the above-mentioned computer executable instructions can be any form of instruction, and the above-mentioned computer program can be any form of application, module, or plugin.

[0077] The embodiments of this application can be implemented using artificial intelligence (AI) technology. AI is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can react in a way similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0078] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Among these, pre-trained models, also known as large-scale models or foundational models, can be widely applied to downstream tasks across various AI fields after fine-tuning. AI software technologies primarily include computer vision, language processing, natural language processing, and machine learning / deep learning.

[0079] See Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may be... Figure 1 Server 200, Figure 2 The server 200 shown includes at least one processor 410, memory 450, and at least one network interface 420. The various components of server 200 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0080] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0081] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0082] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0083] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0084] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0085] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0086] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A data processing device 455 for a virtual scene stored in memory 450 is shown. It can be software in the form of programs and plug-ins, including the following software modules: data acquisition module 4551, data pruning module 4552, and interpretation module 4553. These modules are logical and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0087] The following describes the data processing method for virtual scenes provided in the embodiments of this application. As mentioned above, the electronic device implementing the data processing method for virtual scenes in the embodiments of this application can be a terminal device or a server, or a combination of both. Therefore, the executing entity of each step will not be described again below.

[0088] It should be noted that the examples of virtual scene data processing methods in the following text are illustrated using the live streaming field as an example. Based on the understanding of the following text, those skilled in the art can apply the virtual scene data processing methods provided in the embodiments of this application to data processing in other fields, such as: narration of online videos, narration of cached videos in playback.

[0089] See Figure 3A , Figure 3A This is a flowchart illustrating the data processing method for a virtual scene provided in this application embodiment, which will be combined with... Figure 3A The steps shown are explained. Figure 3A The entity responsible for executing the steps is Figure 1 Server 200.

[0090] In step 301, virtual scene data is acquired.

[0091] For example, virtual scene data includes multiple types of sub-data. To facilitate understanding, the application scenario of the virtual scene data processing method provided in this application embodiment will be explained. The virtual scene can be a game virtual scene. When the screen of the game virtual scene is displayed on the terminal device, for example, watching a video game competition, the user may need corresponding commentary. Commentary content is generated based on the data in the game virtual scene to help the user understand the events occurring in the virtual scene. The events in the virtual scene that need to be explained are the commentary events. The data corresponding to the commentary events is stored in the form of a cache queue, and the difference between the generation time of the data to be explained and the current time is the commentary delay duration.

[0092] The virtual scene data includes several types of sub-data, such as: player character data in the virtual scene (e.g., player character's actions or appearance data, player character winning in a virtual match), non-player character data in the virtual scene (e.g., non-player character's health), player character interaction data with other characters in the virtual scene (e.g., player character defeating other characters), environmental data in the virtual scene (e.g., a certain area in the virtual scene is an ocean, a certain location has a building), and virtual item data in the virtual scene (e.g., player character picking up virtual item A).

[0093] In some embodiments, step 301 can be implemented by: obtaining the virtual scene features of the current virtual scene; identifying each narration event in the virtual scene based on the virtual scene features, wherein the narration event is the content that needs to be narrated in the virtual scene; and combining the data corresponding to each narration event into virtual scene data.

[0094] For example, step 301 can be implemented using a neural network model. The neural network model is invoked to extract features from the current frame data of the virtual scene, obtaining the virtual scene features of the current virtual scene. Based on these features, the neural network model is then invoked for recognition processing to obtain data of type "narrative event," and the data from each narrative event is combined into virtual scene data.

[0095] In step 302, the narration delay duration of the virtual scene data is obtained.

[0096] For example, the commentary latency is the difference between the moment the virtual scene data was generated and the current moment, approximately equal to the waiting time of the game data in the cache queue. Virtual scene data comprises multiple sub-data sets, and the commentary latency of the virtual scene data is a set of sub-commentary latency durations composed of these sub-data sets.

[0097] In some embodiments, step 302 can be implemented in the following ways: obtaining the generation time of each sub-data in the virtual scene data; determining the time difference between the generation time of each sub-data and the current time; using each time difference as the sub-narration delay duration of each sub-data; and combining the sub-narration delay duration of each sub-data into the narration delay duration of the virtual scene data.

[0098] As shown above, the obtained virtual scene data is uninterpreted data, and the generation time of the sub-data of the virtual scene data can be determined by calling the processor running the virtual scene.

[0099] In step 303, feature labels corresponding to multiple types of sub-data are obtained.

[0100] For example, feature labels are used to characterize the importance of each type of sub-data in the narration content, or the amount of information it contains. The feature labels corresponding to each sub-data are generated before acquiring the virtual scene data. These feature labels are pre-stored in corresponding mapping tables or databases, and are directly retrieved from these tables or databases after acquiring the virtual scene data. The higher the importance of a feature label, the higher the narration delay threshold for the sub-data corresponding to that label will be when it is filtered out compared to sub-data corresponding to other feature labels with lower importance.

[0101] For example, if feature label A represents a higher level of importance than feature label B, or represents more information than feature label B, and the current interpretation delay is Tm, the sub-data corresponding to feature label A will not be filtered out, while the sub-data corresponding to feature label B will be filtered out.

[0102] In some embodiments, step 303 can be implemented by querying a first mapping relationship table based on multiple types of sub-data to obtain feature labels corresponding to the multiple types of sub-data respectively, wherein the first mapping relationship table includes: the mapping relationship between each feature label and each type of sub-data.

[0103] For example, in the first mapping table, each feature label can have a one-to-many relationship or a one-to-one relationship with each type of sub-data. In practical applications, this can be pre-configured according to the narration requirements of the game's virtual scene.

[0104] In some embodiments, reference Figure 3F , Figure 3F This is a schematic diagram of the sixth process of the data processing method for virtual scenes provided in the embodiments of this application; in Figure 3A Before step 301, execute Figure 3FSteps 3061 to 3062 determine the first mapping relationship table of feature labels corresponding to multiple types of sub-data, which is explained in detail below.

[0105] In step 3061, the first classification model is called to perform classification processing based on multiple sub-data in the virtual scene to obtain the feature label corresponding to each type of sub-data.

[0106] For example, the first classification model can be a convolutional neural network model, which includes convolutional layers, pooling layers, and fully connected layers. The convolutional and pooling layers are used to extract features from each sub-data set, and the fully connected layers are used to integrate the features obtained from the convolutional and pooling layers into the final output type.

[0107] In some embodiments, step 3061 can be implemented in the following way: based on multiple sub-data in the virtual scene, the first classification model is called to perform the following processing: perform feature extraction processing on the multiple sub-data to obtain multiple sub-virtual scene features; determine the type of each sub-data based on the similarity between the multiple sub-virtual scene features; perform feature classification on the multiple sub-virtual scene features to obtain the feature labels corresponding to each type of sub-data.

[0108] For example, the first classification model is used to determine the type of sub-data and the feature labels corresponding to each type of sub-data. When the similarity between two sub-virtual scene features is less than the type similarity threshold (which can be pre-configured according to the application scenario), it means that the two have low similarity and belong to different types of sub-data. Feature classification is used to determine the importance or information content of the sub-virtual scene features in the narration content.

[0109] In some embodiments, before step 3061, a first classification model is trained to obtain a first sample set, wherein the first sample set includes: multiple sample sub-data, the actual type corresponding to each sample sub-data, and the actual feature label corresponding to each sample sub-data; the first classification model is initialized and classified based on the first sample set to obtain the predicted type corresponding to each sample sub-data and the predicted feature label corresponding to each sample sub-data; a first loss function of the first classification model is determined based on the first difference between the predicted type and the actual type and the second difference between the predicted feature label and the actual feature label; the parameters of the first classification model are adjusted based on the first loss function to obtain the trained first classification model.

[0110] For example, the initialized first classification model has at least the functions of classification and feature extraction. The first loss function can be determined as follows: a first cross-entropy loss is determined based on the first difference between the predicted probability corresponding to the predicted type and the predicted probability corresponding to the actual type; a second cross-entropy loss is determined based on the second difference between the predicted probability of the predicted feature label and the predicted probability of the actual feature label; and the first and second cross-entropy losses are weighted and summed to obtain the first loss function. The sum of the weights corresponding to the first and second cross-entropy losses is 1. The parameters of the first classification model can be adjusted by backpropagation.

[0111] In step 3062, a first mapping table between sub-data of each type and its corresponding feature label is constructed.

[0112] For example, in the first mapping table, the relationships between sub-data and feature labels include the following types: multiple types of sub-data correspond to the same feature label, for example: sub-data corresponding to the interaction event of a player character defeating another player character, and sub-data corresponding to the interaction event of a player character defeating a non-player character, both correspond to the feature label "Important 1"; one type of sub-data corresponds to one feature label, for example: sub-data corresponding to the match result corresponds to the feature label "Important 0". The smaller the number corresponding to the feature label, the higher the importance, and the less likely it is to be filtered out in the presence of commentary delay.

[0113] In this embodiment of the application, before displaying the screen to be explained in the virtual scene, a first mapping table between each type of sub-data and the corresponding feature label is constructed, so that existing feature labels can be directly called during the explanation process, saving the computing resources and time required to obtain the explanation content, improving response efficiency, and alleviating explanation delay.

[0114] Continue to refer to Figure 3A In step 304, based on the feature labels of each type of sub-data and the interpretation delay duration, at least one type of sub-data is filtered out from multiple types of sub-data to obtain the data to be interpreted.

[0115] For example, feature labels represent the information content or importance of sub-data of a certain type in a virtual scene. Different feature labels correspond to different second time intervals. When the interpretation delay of the sub-data corresponding to a feature label falls within the second time interval corresponding to that feature label, the sub-data corresponding to that feature label is filtered out. This process is repeated for each sub-data, and the remaining sub-data obtained after filtering is used as the data to be interpreted.

[0116] In some embodiments, the interpretation delay duration includes: sub-interpretation delay durations corresponding to each of the multiple types of sub-data; reference Figure 3B , Figure 3B This is a schematic diagram of the second process of the data processing method for virtual scenes provided in the embodiments of this application; Figure 3A Step 304 can be achieved through Figure 3B Steps 3041 to 3043 are implemented, and the details are explained below.

[0117] In step 3041, the target first time interval in which the sub-explanation delay time of each sub-data is located is determined from multiple first time intervals preset for each feature label.

[0118] For illustrative purposes, the relationships between sub-data in the embodiments of this application are explained below with reference to the accompanying drawings. Figure 5 , Figure 5 This is a schematic diagram illustrating the principle of frame dropping processing provided in this application embodiment; new data (sub-data) generated in the virtual scene will first be cached at the tail position of the game data cache queue 501. Figure 1 The interpretation calculation module 503 in server 200 sequentially extracts data from the head of the game data cache queue 501 (e.g., head data 502). Since the generation time of sub-data may not be the same, the sub-interpretation delay time of a batch of uninterpreted virtual scene data may be different. In this case, the sub-interpretation delay time of each sub-data is judged separately.

[0119] For example, for each sub-data set, multiple preset first duration intervals are traversed for each feature label corresponding to the sub-data set. When the sub-explanation delay duration of the sub-data set falls within one of these first duration intervals, that first duration interval is selected as the target first duration interval. The preset multiple first duration intervals are multiple adjacent time intervals, each with a duration that may be equal or unequal. The upper and lower limits of the first duration intervals are set according to the actual application scenario. Different first duration intervals for the same feature label can be used to represent different levels of delay.

[0120] In some embodiments, reference Figure 3C , Figure 3C This is a schematic diagram of the third process of the data processing method for a virtual scene provided in the embodiments of this application; Figure 3B Step 3041 can be achieved through Figure 3C Steps 30411 to 30413 are implemented, and the details are explained below.

[0121] In step 30411, the decision tree model is obtained.

[0122] For example, the decision tree model includes multiple root nodes and decision paths corresponding to each root node. The root node represents the feature label of the sub-data. Each decision path includes multiple leaf nodes. The leaf nodes represent a first time interval corresponding to the feature label of each sub-data. The first time intervals are sequentially adjacent, and the upper limit of the first time interval of each leaf node is the lower limit of the next first time interval.

[0123] For example, each first time interval represents a different level of latency. For different feature labels, the upper and lower bounds of multiple first time intervals can be different. (Reference) Figure 8 , Figure 8 This is a schematic diagram of the second structure of the decision tree model provided in the embodiments of this application. The first root node of the decision tree model represents different feature labels of the virtual scene data 800, for example: root node 810 (referring to feature label 0), root node 811 (referring to feature label 1), root node 812 (referring to feature label 2), and so on up to root node N (referring to feature label N). N is a positive integer greater than 2.

[0124] In step 30412, for each type of sub-data, the decision path corresponding to the type of sub-data in the decision tree model is traversed.

[0125] For example, the decision path of the decision tree model can determine the first time interval of the sub-interpretation delay corresponding to the sub-data based on the judgment conditions in the nodes included in the path.

[0126] In step 30413, in response to the sub-explanation delay duration of the sub-data of the type being located in the first duration interval of the first leaf node traversed, the first duration interval in which the sub-explanation delay duration is located is taken as the target first duration interval of the sub-data of the type.

[0127] For example, in a decision path, when the first leaf node reached satisfies the following condition, the sub-interpretation delay duration of the sub-data of the type is in the first duration interval of the first leaf node reached. Stop the traversal processing for the decision path and output the target first duration interval in which the sub-interpretation delay duration represented by the first leaf node reached satisfies the condition.

[0128] Continue to refer to Figure 8Taking decision path 830 of root node 810 as an example, each leaf node of decision path 830 is used to determine the first time interval to which the interpretation delay belongs. For example, the judgment condition corresponding to leaf node 8101 is "t < T1", where t is the interpretation delay. If the interpretation delay t is greater than T1, the judgment proceeds to leaf node 8102, and so on. If the interpretation delay t is less than T1, the target first time interval represented by leaf node 8201, "target interval 0", is output. Leaf nodes 8102, 8103, and 8104 correspond to different first time intervals, and leaf nodes 8202, 8203, and 8204 correspond to different target first time intervals. The judgment process between leaf nodes is the same as that of leaf node 8101, and will not be repeated here.

[0129] The following are examples of decision tree models applied in specific scenarios. (For reference...) Figure 6 , Figure 6 This is a first structural diagram of the decision tree model provided in the embodiments of this application. Figure 6 The decision tree model is Figure 8 This is one implementation of the decision tree model in a game application scenario. The root nodes of the decision tree model are labels representing different levels of importance in the game data 600, for example: root node 610 (representing importance 0), root node 611 (representing importance 1), root node 612 (representing importance 2), root node 613 (representing importance 3), up to root node N (representing importance N), where N is a positive integer greater than 3. Each root node corresponds to a different decision path. Taking decision path 630 of root node 610 as an example, decision path 630 includes multiple sequentially connected leaf nodes, each corresponding to a different judgment condition or a different judgment result. In decision path 630, leaf node 6101 is used to determine whether the current explanation delay is less than T1. If it is less than T1, the delay level represented by leaf node 6201, "delay 0," is output; otherwise, the process proceeds to the judgment process corresponding to leaf node 6102. Leaf node 6102 is used to determine whether the current narration delay is less than T2. ​​If it is less than T2, the delay level "Delay 1" represented by leaf node 6202 is output; otherwise, the process proceeds to the judgment process corresponding to leaf node 6103. Leaf node 6103 is used to determine whether the current narration delay is less than T3. If it is less than T3, the delay level "Delay 2" represented by leaf node 6203 is output; otherwise, the process proceeds to the judgment process corresponding to leaf node 6104. Leaf node 6104 is used to determine whether the current narration delay is less than T4. If it is less than T4, the delay level "Delay 3" represented by leaf node 6204 is output; otherwise, the process proceeds to the judgment process corresponding to the subsequent leaf nodes. This process continues until the corresponding delay level is obtained.

[0130] In this embodiment of the application, the first time interval of the target is determined by a decision tree model. Compared with traversing all the sub-data in a sequential manner, this can save the time and computing resources required for traversal, and improve the accuracy and efficiency of obtaining the first time interval of the target.

[0131] Continue to refer to Figure 3B In step 3042, the second time interval corresponding to each feature label is obtained.

[0132] For example, the second duration interval of a feature label includes a portion of the first duration interval corresponding to the feature label; or the duration of the second duration interval is zero. When the duration of the second duration interval of a feature label is zero, it means that the sub-data corresponding to that feature label will not be filtered out under any commentary delay. For example, if the game result is very important in game commentary, then the second duration interval of the feature label corresponding to the game result is 0, and the game result will not be filtered out under any commentary delay. The second duration interval corresponding to each feature label is generated before the virtual scene being commented on runs and is directly retrieved from the storage structure during the commentary process. The storage structure can be a mapping table or a tree model.

[0133] In some embodiments, Figure 3A Before step 301, refer to Figure 3G , Figure 3G This is a schematic diagram of the seventh process of the data processing method for virtual scenes provided in the embodiments of this application; execution Figure 3G Steps 3071 to 3074 determine the second time interval of the feature labels corresponding to the multiple types of sub-data, as explained in detail below.

[0134] In step 3071, the second classification model is called to perform the following processing based on each feature label: perform probability prediction processing on the feature label to obtain the predicted probability of the feature label for different first time intervals.

[0135] For example, the second classification model is a model with different functions from the first classification model mentioned above. The second classification model can also be a convolutional neural network (CNN) model, which includes convolutional layers, pooling layers, and fully connected layers. The convolutional and pooling layers are used to extract features from each sub-data set, and the fully connected layers are used to integrate the features processed by the convolutional and pooling layers into the final output prediction probability. The prediction probability represents the probability that the first time interval belongs to the second time interval.

[0136] In some embodiments, before step 3071, a second classification model can be trained in the following manner to obtain a second sample set, wherein the second sample set includes: sample feature labels corresponding to multiple sample sub-data and actual second time intervals corresponding to each sample feature label; the initialized second classification model is called based on the second sample set to perform classification processing, thereby obtaining the predicted second time intervals corresponding to the sample feature labels of each sample sub-data; a second loss function of the second classification model is determined based on the third difference between the predicted second time interval and the actual second time interval; and the parameters of the second classification model are adjusted based on the second loss function to obtain the trained second classification model.

[0137] For example, the training principle of the second classification model can be referred to that of the first classification model, and will not be repeated here.

[0138] In step 3072, in response to the fact that the number of first time intervals that meet the classification conditions is one, the first time interval that meets the classification conditions is used as the second time interval of the feature label.

[0139] For example, the classification criteria include: the predicted probability corresponding to the first time interval is greater than a probability threshold. The probability threshold is set according to the actual needs of the application scenario. When only one first time interval meets the classification criteria, it means that the first time interval is the second time interval.

[0140] In step 3073, in response to the fact that there are multiple first time intervals that meet the classification conditions, the earliest and latest times among the multiple first time intervals that meet the classification conditions are determined, and the time interval between the earliest and latest times is used as the second time interval of the feature label.

[0141] For example, suppose the first time intervals satisfying the classification conditions are: "T2≤t<T3", "T4≤t<T5", and "T5≤t<T6". Among these three intervals, "T2≤t<T3" and "T4≤t<T5" are not adjacent, while "T4≤t<T5" and "T5≤t<T6" are adjacent. The time interval between the earliest time T2 and the latest time T6 is taken as the second time interval of the feature label, represented as "T2≤t<T6".

[0142] In step 3074, in response to the absence of a first time interval that satisfies the classification conditions, it is determined that when the interpretation delay time is any value, the sub-data corresponding to the feature label are all data to be interpreted.

[0143] For example, if there is no first time interval that meets the first classification condition, it means that the sub-data needs to be retained under any circumstances and is used as data to be explained.

[0144] In this embodiment of the application, by determining the second time interval that needs to be filtered out for each feature label in the above manner, the corresponding second time interval can be determined according to the importance or information content of the sub-data. During the explanation process, the generated second time interval can be directly called, thereby improving the response efficiency and explanation effect of the explanation process.

[0145] Continue to refer to Figure 3B In step 3043, based on the second time interval of each feature label and the first time interval of the target, the target type sub-data is determined among multiple types of sub-data, and the target type sub-data is filtered out from the multiple types of sub-data to obtain the data to be explained.

[0146] For example, the target type sub-data is the sub-data corresponding to the feature labels of the target in the first time interval that fall within the second time interval. When the storage structure is a mapping table, the target type sub-data can be determined by fully traversing each sub-data and then filtered out sequentially. When the storage structure is a tree model, the target type sub-data can be filtered out by pruning. The following provides a detailed explanation of each method.

[0147] In some embodiments, the second duration interval is integrated into a computational tree model; see reference. Figure 3D , Figure 3D This is a schematic diagram of the fourth process of the data processing method for a virtual scene provided in the embodiments of this application; Figure 3B Step 3042 can be achieved through Figure 3D Step 30421A is implemented, and the details are explained below.

[0148] In step 30421A, the computation tree model is obtained.

[0149] For example, the multiple second leaf nodes of the computational tree model include a second time interval associated with each sub-data. The computational tree model is a statistical model; in this embodiment, it is used to store the second time interval of the sub-data and to filter out sub-data by traversing the computational tree model.

[0150] Continue to refer to Figure 3D , Figure 3B Step 3043 can be achieved through Figure 3D Steps 30431A to 30433A are implemented, and the details are explained below.

[0151] In step 30431A, the decision paths corresponding to the second root node of the computation tree model are traversed.

[0152] For example, the second root node represents the feature label of the sub-data. Each decision path includes multiple second leaf nodes. The multiple second leaf nodes in the same decision path are sorted in ascending order according to the number of first time intervals included in the corresponding second time interval.

[0153] For ease of understanding, please refer to Figure 9 , Figure 9 This is a schematic diagram of the second structure of the computational tree model provided in this application embodiment. The second root node of the computational tree model is a different feature label of the virtual scene data 900, including: root node 910 (feature label 0), root node 911 (feature label 1), root node 912 (feature label 2), etc. Taking the decision path 920 of root node 910 as an example, decision path 920 includes leaf nodes 9101, 9102, 9103, 9104, etc. The number of first time intervals included in the second time intervals corresponding to leaf nodes 9101 to 9104 increases sequentially. Leaf node 9101 represents "sub-data 1 is not pruned", that is, the second time interval of leaf node 9101 includes 0 first time intervals.

[0154] In step 30432A, in response to the second leaf node being traversed satisfying the filtering condition, the following sub-data are filtered out: the sub-data corresponding to the target second leaf node that satisfies the filtering condition, and the sub-data of each subsequent second leaf node connected to the target second leaf node.

[0155] For example, the filtering criteria include: the target first time interval of the feature label is located in the second time interval corresponding to the feature label. For example, in the computational tree model, the second leaf nodes in each decision path are sorted in ascending order according to the length of the second time interval. In response to the current node being pruned, the traversal of the decision path is stopped, and the current node and subsequent nodes are deleted.

[0156] Continue to refer to Figure 9 Each leaf node in decision path 920 is traversed. When traversing to leaf node 9103, if the target's first time interval is within the second time interval of leaf node 9103, then the sub-data corresponding to leaf node 9103, leaf node 9104, and each subsequent leaf node are filtered out. The sub-data of leaf nodes 9101 and 9102 in decision path 920 are retained and used as data to be explained.

[0157] In step 30433A, in response to the end of the traversal process, the remaining data in the multiple types of sub-data are taken as the data to be explained.

[0158] For example, the traversal of different decision paths can be performed synchronously or sequentially. When the traversal of all decision paths is completed, the remaining sub-data is used as the data to be interpreted.

[0159] The following are examples of computation tree models applied in specific scenarios. (For reference...) Figure 7 , Figure 7 This is a schematic diagram of the structure of the computation tree model provided in the embodiments of this application. Figure 7 The computation tree model is Figure 9 This paper describes one implementation of the computational tree model in a game application scenario. The root node of the computational tree model represents game features corresponding to different levels of importance in game data 700, including: root node 710 (representing player features), root node 711 (representing non-player character features), root node 712 (representing player events), ..., root node 71N, where N is a positive integer. In this embodiment, game commentary in an e-sports competition is used as an example for illustration. Player features are the characteristics of the player character controlled by the player, and player events are events generated by the player character controlled by the player in the game scene.

[0160] For example, each root node corresponds to a different decision path, and each decision path includes multiple leaf nodes. Each leaf node represents a different judgment condition. If the latency of the game data satisfies the corresponding judgment condition, then the game data of the leaf node that satisfies the condition, as well as the game data corresponding to the leaf nodes after that leaf node, are filtered out. The remaining game data is then used as the data to be explained. (Continue to refer to...) Figure 7 The leaf nodes corresponding to root node 712 include leaf nodes 7121, 7122, and 7123. Leaf node 7121 indicates that player defeat events will not be filtered out from the game data corresponding to the feature labels of player events. Player defeat events are events where a player-controlled character defeats another player character, or defeats a non-player character. Leaf node 7122 indicates that player status events will be pruned at latency levels 3 to N. That is, if the latency of the game data for player status events falls within the range of latency levels 3 to N, the game data for player status events will be pruned, as will the game data for "player performance events" corresponding to leaf node 7123. Taking decision path 720 of leaf node 710 as an example for explanation, the game data represented by "player score-related features" of leaf node 7101 will not be pruned at any latency level. If the current latency level is within any of the latency levels of leaf nodes 7102, 7103, and 7104, then the game data corresponding to the corresponding leaf node and its subsequent leaf nodes will be filtered out. When each decision path in the computational tree model has been traversed, the remaining game data in the current frame will be used as the data to be interpreted.

[0161] In this embodiment, data filtering is performed based on a computational tree model. The nodes of the computational tree model record the time interval for filtering the interpretation delay of each sub-data, thereby improving the efficiency of data filtering.

[0162] In some embodiments, reference Figure 3E , Figure 3E This is a schematic diagram of the fifth process of the data processing method for a virtual scene provided in the embodiments of this application; Figure 3B Step 3042 can be achieved through Figure 3E Step 30421B is implemented, and the details are explained below.

[0163] In step 30421B, the second mapping relationship table is queried based on each feature label to obtain the second time interval corresponding to each feature label.

[0164] For example, the second mapping table includes the mapping relationship between each feature label and its corresponding second time interval. This second mapping table is constructed before acquiring the virtual scene data.

[0165] Continue to refer to Figure 3E , Figure 3B Step 3043 can be achieved through Figure 3E Steps 30431B to 30433B are implemented, and the details are explained below.

[0166] In step 30431B, each feature label is traversed.

[0167] For example, the sub-data associated with each feature label is traversed in the order of the serial number corresponding to each feature label. Each feature label corresponds to at least one sub-data segment. Traversing the sub-data segment by feature label can improve traversal efficiency.

[0168] In step 30432B, in response to the fact that the target first time interval of the traversed feature label is in the second time interval corresponding to the feature label, it is determined that the type of the sub-data corresponding to the traversed feature label is the target type, and the sub-data of the target type is filtered out.

[0169] For example, during the traversal, the sub-data that "the first time interval of the target is in the second time interval corresponding to the feature label" is filtered out in turn. When the traversal is completed, all data of the target type that needs to be filtered out can be filtered out.

[0170] In step 30433B, in response to the end of the traversal process, the remaining data in the multiple types of sub-data are taken as the data to be explained.

[0171] For example, the principle of step 30433B is the same as that of step 30433A, and will not be repeated here.

[0172] In this embodiment, the sequential traversal method can improve the accuracy of filtering sub-data, avoid erroneous filtering, maintain the effect of virtual scene narration, and improve the user's listening experience.

[0173] Continue to refer to Figure 3A In step 305, the narration content of the virtual scene is generated based on the data to be narrated.

[0174] For example, the narration can be presented to the audience in any of the following ways: audio; audio combined with subtitles; subtitles.

[0175] In some embodiments, step 305 can be implemented in the following ways: calling a neural network model to perform text generation processing based on the data to be explained to obtain the explanation text; calling a neural network model to perform speech generation processing based on the explanation text to obtain the explanation speech; and using the explanation speech as the explanation content of the virtual scene.

[0176] For example, the neural network model can be a large language model. Speech generation processing can be based on each character in the narration text, querying to obtain the pre-configured pronunciation of each character, combining the pronunciations of each character to obtain an audio sequence, and calling the neural network model to insert intervals and adjust the playback speed in the audio sequence to simulate punctuation and speech rate in real human speech.

[0177] In some embodiments, this application also proposes a data processing method for virtual scenes, to... Figure 1 The terminal device in the process acts as the execution entity, displaying a virtual scene in the human-computer interaction interface; and playing narration content corresponding to the virtual scene, wherein the narration content is generated based on the virtual scene data processing method provided in the embodiments of this application. The narration content may be... Figure 1 The terminal device 400 itself, or Figure 1 It was generated by server 200.

[0178] In this embodiment, sub-data is filtered out based on the narration delay duration in the virtual scene and the feature tags of each sub-data. Compared with the full narration scheme in related technologies, this simplifies the data to be narrated in the virtual scene, resulting in a shorter delay between the narration content generated based on the data to be narrated and the current virtual scene screen, improving the smoothness of the virtual scene narration and saving the computing resources consumed in generating the virtual scene narration content. Compared with the scheme in related technologies that directly drops frames from the virtual scene data based on the narration delay duration, this method uses the feature tags of the sub-data and the narration delay duration as the judgment criteria to determine the data to be narrated, avoiding the loss of important content in the narration and improving the user's experience of listening to the virtual scene narration.

[0179] The following will describe an exemplary application of the data processing method for virtual scenes according to the embodiments of this application in a real-world application scenario.

[0180] Currently, video games have become a popular product, and correspondingly, game AI commentary is receiving increasing attention. High-quality AI commentary is a crucial component of a game's vitality. Commentary latency is the delay between the moment an event occurs in the game and the moment that event is narrated. Commentary latency is one of the most important indicators for evaluating commentary quality. When the game's frame rate is low, optimizing the processing time of a single frame can effectively solve the latency problem. However, with the increasingly higher frame rates of mainstream games and the higher network resources required to run virtual game scenes, the commentary latency problem becomes particularly prominent.

[0181] Currently, the main solution to the commentary latency issue is a frame dropping solution. In this solution, to ensure the stability of the game commentary played on the terminal device, the backend server introduces a data caching queue, as shown in the reference... Figure 5 , Figure 5 This is a schematic diagram illustrating the principle of frame dropping processing provided in this application embodiment. New data is first cached at the tail of the game data cache queue 501. The commentary calculation module 503 in the server sequentially extracts data from the head of the game data cache queue 501 (e.g., head data 502) and performs calculations to obtain AI real-time commentary content. If the waiting time of the extracted head data in the cache queue (i.e., timeout judgment) exceeds a set threshold, this frame of game data will be directly discarded and will not be extracted by the commentary calculation module to generate AI real-time commentary content. This process is defined as frame dropping. Conversely, if the waiting time of the extracted head data in the cache queue does not exceed the set threshold, the commentary calculation module 503 generates commentary content based on the head data.

[0182] When the AI ​​commentary module's processing power is insufficient, game data accumulates in the cache queue, causing commentary latency. Using a frame dropping scheme to quickly discard delayed data can catch up with the actual game progress as soon as possible, alleviating the latency issue between the commentary and the game content. However, this scheme discards some game data frames, potentially missing key event announcements and degrading the commentary experience. Furthermore, the frame dropping scheme cannot automatically adapt to the game's frame rate; when the frame rate increases, a large amount of new data is generated. Theoretically, optimizing the single-frame processing time of the commentary module can reduce the number of dropped frames, but performance optimization of the commentary module has its limitations. The latency relief achieved by optimizing the commentary module's performance is often less than the pressure from the increased data volume brought about by higher frame rates.

[0183] To address the aforementioned problems in related technologies, this application's embodiments can dynamically prune the data calculation based on the current characteristics of the game data and the real-time commentary latency. This selectively filters out some data while retaining some data to be commented on based on the game data's characteristics, maintaining the commentary effect while reducing latency. This process is defined as dynamic pruning calculation. Dynamic pruning calculation ensures that no crucial game data is lost and no key game event broadcasts are missed. Furthermore, it can automatically adjust the pruning range based on the current game latency, ensuring that the game commentary latency remains at a relatively low level. Dynamic pruning calculation can also adapt to changes in the game frame rate. In different game scenarios, only the relevant parameters need to be adjusted according to the actual needs of the virtual scene data, demonstrating generalization ability.

[0184] refer to Figure 4 , Figure 4 This is a schematic diagram of the eighth process of the data processing method for virtual scenes provided in the embodiments of this application; it will be combined with Figure 4 The steps shown are explained. Figure 4 The entity responsible for executing the steps is Figure 1 Server 200.

[0185] In step 401, the game data is classified according to its characteristics to obtain the importance of different types of game data.

[0186] For example, the importance of each frame of game data varies, and different game data can be categorized based on the actual game scenario. The final classification of each type of game data may differ, and the importance is distinguished from the perspective of AI commentary. The importance of game data (as mentioned above) is determined based on its importance within the AI ​​commentary content.

[0187] In some embodiments, the importance of game data can be set manually. For example, users vote according to their attention to different commentary events in the game commentary, and the descending order of the number of votes corresponding to each commentary event is used as a feature label of the importance of the game data associated with each commentary event.

[0188] For example, before game live streaming, the data of the game being streamed is categorized by importance. The sample game has a limited number of game data types (e.g., 100). The importance of different game data types is used as feature labels to train a first classification model. This model can perform the following processes: extract features from all game data of the game to be streamed, obtain game data features for all game data, determine the different types of game data based on these features, and classify each data type by importance to obtain the importance of each type of game data. In other words, the first classification model has the function of determining the types of game data for different types and the importance of different data types. In this embodiment, the first classification model trained as described above can enhance versatility. The game used in this embodiment is a team-based competitive game; the above method can be applied to other types of games.

[0189] For ease of understanding, the following examples illustrate feature labels of varying importance:

[0190] Important 0: The most important game data. It calls the commentary calculation module to perform full calculations without any pruning. For example, the data representing the result of a match in the live broadcast can be the data of events such as "a player defeating another player in the game".

[0191] Importance 1: Less important than "Importance 0", it allows dynamic pruning calculations, but the range of delay time intervals for pruning is smaller. For example, data that affects the course of a game during a live stream could be data from events such as "players picking up equipment in the game".

[0192] Important 2: Less important than "Important 1", allows dynamic pruning calculations, and the range of delay time intervals for pruning is larger than that of "Important 1", such as data on events like "players changing their posture in the game".

[0193] Important 3: Less severe than "Important 2", allows dynamic pruning calculations, and the range of delay time intervals for pruning is larger than that of "Important 2", such as data of events like "players changing their current coordinates in the game".

[0194] ...

[0195] Importance N: Less important than the "Importance N-1" category, allows for dynamic pruning calculations, and has the largest range of delay intervals for which pruning is performed compared to other importance categories. N is a positive integer greater than 1.

[0196] In step 402, the latency intervals corresponding to different levels of importance of game data are configured.

[0197] For example, latency can be characterized by latency duration intervals (the second duration interval mentioned above). The waiting time for each frame of game data in the cache queue is different and changes dynamically. Therefore, it is necessary to determine the current latency situation and classify the latency level for each frame. The logic for classifying the latency level of game data of each importance is different and depends on the set time threshold parameter. Theoretically, more classifications can control the pruning granularity more precisely, but the corresponding pruning logic is more complex and needs to be determined based on the actual situation.

[0198] For example, if the game has few data types, experience can be summarized through manual experiments to determine the latency range that needs to be pruned for each type of game data. The latency range that needs pruning means that when the current commentary latency reaches the pruning latency range corresponding to that game data, the AI ​​commentary calculation module will not perform calculations on that game data; that is, it will not provide commentary on the content corresponding to that game data. Users listening to the commentary can see the visual representation of that game data in the virtual scene, but will not receive the corresponding commentary content.

[0199] In some embodiments, the latency interval corresponding to each game data type of the game to be explained can be determined by inputting the game data types of the game to be explained into a second classification model, which then obtains the pruned latency interval corresponding to each game data type. The training data for training the second classification model includes: the type of sample game data, the importance of each type of sample game data, and the actual latency interval labeled for different types of sample game data.

[0200] To facilitate understanding, the following examples illustrate different duration intervals for game data with an importance level of N:

[0201] Assumptions: For game data categorized as "Important N", the interpretation latency of the current game frame is time T. Latency threshold parameters T1-Tn are set, and latency levels are categorized as follows:

[0202] Delay 0: Delay time T < T1, the current interpretation delay is very low, and full calculation can be performed in the interpretation calculation module without any pruning.

[0203] Delay 1: T1 ≤ delay time T < T2. The current explanation has a low delay, which allows for dynamic pruning calculations, but the pruning range is small.

[0204] Delay 2: T2 ≤ Delay time T < T3, the current interpretation has a delay, dynamic pruning calculation is allowed, the pruning range is larger than "Delay 1" (that is, the range of data that can be dropped is larger than Delay 1).

[0205] Delay 3: T3 ≤ Delay time T < T4, the current interpretation has a delay, dynamic pruning calculation is allowed, the pruning range is larger than "Delay 2" (that is, the range of data that can be dropped is larger than Delay 2).

[0206] ...

[0207] Delay N: Tn ≤ delay time T. The current interpretation has extremely high delay, allowing for dynamic pruning calculations. The pruning range is the largest compared to other delay levels. N and n are both positive integers.

[0208] Different delay time threshold parameters T1-Tn can be configured for data of different importance levels, and the T1-Tn parameters can be inconsistent for data of different importance levels.

[0209] In step 403, the current commentary delay of the game data is obtained, the game data is filtered, and the data to be commented on is obtained.

[0210] For example, based on the game data of the current frame, a decision tree is invoked to determine the target duration interval corresponding to the current commentary delay. The delay interval to be pruned varies depending on the importance of the data. If the target duration interval of the game data falls within the range of the pruned delay interval, then no commentary is provided for that game data; that is, the game data is pruned. The remaining unpruned data is used to generate commentary content. The principle of the decision tree model is explained below with reference to the accompanying diagram. Figure 6 , Figure 6This is a schematic diagram of the decision tree model provided in this application embodiment. The root nodes of the decision tree model are different importance labels of game data 600, including root node 610 (representing importance 0), root node 611 (representing importance 1), root node 612 (representing importance 2), root node 613 (representing importance 3), up to root node N (representing importance N), where N is a positive integer greater than 3. Each root node corresponds to a different decision path. Taking the decision path 630 of root node 610 as an example, decision path 630 includes multiple sequentially connected leaf nodes, each corresponding to a different judgment condition or a different judgment result. In decision path 630, leaf node 6101 is used to determine whether the current explanation delay time is less than T1. If it is less than T1, the delay level represented by leaf node 6201, "delay 0", is output. If not, the judgment processing corresponding to leaf node 6102 is initiated. Leaf node 6102 is used to determine whether the current narration delay is less than T2. ​​If it is less than T2, the delay level "Delay 1" represented by leaf node 6202 is output; otherwise, the process proceeds to the judgment process corresponding to leaf node 6103. Leaf node 6103 is used to determine whether the current narration delay is less than T3. If it is less than T3, the delay level "Delay 2" represented by leaf node 6203 is output; otherwise, the process proceeds to the judgment process corresponding to leaf node 6104. Leaf node 6104 is used to determine whether the current narration delay is less than T4. If it is less than T4, the delay level "Delay 3" represented by leaf node 6204 is output; otherwise, the process proceeds to the judgment process corresponding to the subsequent leaf nodes. This process continues until the corresponding delay level is obtained.

[0211] Based on the latency levels corresponding to the game data, the explanation calculation module calls the computation tree model to perform dynamic pruning according to the latency levels. (See reference...) Figure 7 , Figure 7 This is a schematic diagram of the computational tree model provided in this application embodiment. The root node of the computational tree model represents game features corresponding to different levels of importance of game data 700, including: root node 710 (representing player features), root node 711 (representing non-player character features), root node 712 (representing player events), ... root node 71N, where N is a positive integer. In this application embodiment, game commentary in an e-sports competition is used as an example for illustration. Player features are the features of the player character controlled by the player, and player events are events generated by the player character controlled by the player in the game scene.

[0212] For example, each root node corresponds to a different decision path, and each decision path includes multiple leaf nodes. Each leaf node represents a different judgment condition. When the latency of the game data satisfies the corresponding judgment condition, the game data of the leaf node that satisfies the condition, as well as the game data corresponding to the leaf nodes after that leaf node, are filtered out. The remaining game data is then used as the data to be explained. (Continue to refer to...) Figure 7 The leaf nodes corresponding to root node 712 include leaf nodes 7121, 7122, and 7123. Leaf node 7121 indicates that player defeat events will not be filtered out from the game data corresponding to the feature labels of player events. Player defeat events are events where a player-controlled character defeats another player character, or defeats a non-player character. Leaf node 7122 indicates that player status events will be pruned at latency levels 3 to N. That is, if the latency of the game data for player status events falls within the range of latency levels 3 to N, the game data for player status events will be pruned, as will the game data for "player performance events" corresponding to leaf node 7123. Taking decision path 720 of leaf node 710 as an example for explanation, the game data represented by "player score-related features" of leaf node 7101 will not be pruned at any latency level. If the current latency level is within any of the latency levels of leaf nodes 7102, 7103, and 7104, then the game data corresponding to the corresponding leaf node and its subsequent leaf nodes will be filtered out. When each decision path in the computational tree model has been traversed, the remaining game data in the current frame will be used as the data to be interpreted.

[0213] To facilitate understanding, the following example illustrates the process: For the current frame of game data, the commentary calculation module calls the computation tree model to calculate game features, commentary events, etc. in sequence. Assuming the latency of the current frame of game data is "0", the data to be commented on includes the following: Calculated player features: player score-related features, player equipment-related features, player posture-related features, player coordinate-related features, etc.; Calculated player events: player defeat events, player status events, player performance events, etc.

[0214] When the game frame latency is "Latency 2", the data to be explained includes the following: Calculated player characteristics: player score-related characteristics, player equipment-related characteristics; Calculated player events: player defeat events, player status events.

[0215] As illustrated in the example above, "Delay 2" reduces the calculation of some features and events compared to "Delay 0." The reduced portion is the pruned part. That is, based on the different delay time intervals corresponding to the importance of different data and the interval corresponding to the current commentary delay, some data is pruned, and the commentary content is generated based on the remaining game data. Through this dynamic pruning calculation, the computational load of the commentary calculation module can be reduced accordingly when the commentary delay is relatively large, so as to quickly reduce the commentary delay.

[0216] In step 404, the explanatory content is generated based on the data to be explained.

[0217] For example, based on the data to be explained, a large language model or neural network model is invoked to generate text-based explanations, which can then be presented to viewers of the live stream via audio or subtitles.

[0218] In some embodiments, the virtual scene data processing method provided in this application is applied to e-sports competitions at various levels. Artificial intelligence commentary can effectively replace professional commentary, providing the best viewing experience for players at all levels, while also saving manpower for event organizers.

[0219] In this embodiment, dynamic pruning calculations are performed based on the game's real-time frame rate to ensure that the game commentary latency remains at a low level. It can adapt to changes in the game's frame rate, not just a single frame rate. Based on the characteristics of the game data and the real-time commentary latency, dynamic pruning of the game commentary is implemented. When the commentary latency increases, some less important calculations are pruned away, while important calculations are retained. This ensures low commentary latency without missing key events. It can automatically adapt to changes in the game's frame rate; when the frame rate increases, the pruned calculations increase accordingly, ensuring that the commentary latency remains at a low level.

[0220] The following description continues to illustrate the exemplary structure of the virtual scene data processing device 455 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 2 As shown, the software modules in the virtual scene data processing device 455 stored in the memory 450 may include: a data acquisition module 4551, used to acquire virtual scene data, wherein the virtual scene data includes multiple types of sub-data; the data acquisition module 4551 is also used to acquire the narration delay duration of the virtual scene data and the feature labels corresponding to the multiple types of sub-data respectively; a data pruning module 4552, used to filter out at least one type of sub-data from the multiple types of sub-data based on the feature labels and narration delay duration of each type of sub-data to obtain the data to be narrated; and a narration module 4553, used to generate narration content for the virtual scene based on the data to be narrated.

[0221] In some embodiments, the interpretation delay duration includes: sub-interpretation delay durations corresponding to multiple types of sub-data respectively; the data pruning module 4552 is used to determine the target first time interval where the sub-interpretation delay duration of each sub-data is located from multiple preset first time intervals for each feature label; obtain the second time interval corresponding to each feature label respectively, wherein the second time interval of the feature label includes a portion of the first time interval corresponding to the feature label; based on the second time interval of each feature label and the target first time interval, determine the target type of sub-data among the multiple types of sub-data, and filter out the target type of sub-data from the multiple types of sub-data to obtain the data to be interpreted.

[0222] In some embodiments, the data pruning module 4552 is used to obtain a decision tree model, wherein the decision tree model includes multiple first root nodes and decision paths corresponding to each first root node, the first root node represents the feature label of the sub-data, each decision path includes multiple first leaf nodes, the first leaf nodes represent a first duration interval corresponding to the feature label of each sub-data; for each type of sub-data, the decision paths corresponding to the sub-data of the type in the decision tree model are traversed; in response to the sub-explanation delay duration of the sub-data of the type being in the first duration interval of the traversed first leaf node, the first duration interval in which the sub-explanation delay duration is located is taken as the target first duration interval of the sub-data of the type.

[0223] In some embodiments, the data pruning module 4552 is used to obtain a computational tree model, wherein the multiple second leaf nodes of the computational tree model include a second time interval associated with each sub-data; the data pruning module 4552 is used to traverse the decision path corresponding to the second root node of the computational tree model, wherein the second root node represents the feature label of the sub-data, each decision path includes multiple second leaf nodes, and the multiple second leaf nodes in the same decision path are sorted in ascending order according to the number of first time intervals included in the corresponding second time interval; in response to the traversed second leaf node satisfying the screening condition, the following sub-data are screened out: sub-data corresponding to the target second leaf node that satisfies the screening condition, and sub-data of each subsequent second leaf node connected to the target second leaf node, wherein the screening condition includes: the target first time interval of the feature label is located in the second time interval corresponding to the feature label; in response to the end of the traversal process, the remaining data in the multiple types of sub-data are taken as data to be interpreted.

[0224] In some embodiments, the data pruning module 4552 is used to query a second mapping relationship table based on each feature label to obtain a second time interval corresponding to each feature label, wherein the second mapping relationship table includes: a mapping relationship between each feature label and its corresponding second time interval; the data pruning module 4552 is used to traverse each feature label; in response to the target first time interval of the traversed feature label being located in the second time interval corresponding to the feature label, the type of the sub-data corresponding to the traversed feature label is determined to be the target type, and the sub-data of the target type is filtered out; in response to the end of the traversal process, the remaining data in the sub-data of multiple types is taken as the data to be interpreted.

[0225] In some embodiments, the data acquisition module 4551 is used to acquire the virtual scene features of the current virtual scene; identify each narration event in the virtual scene based on the virtual scene features, wherein the narration event is the content that needs to be narrated in the virtual scene; and combine the data corresponding to each narration event into virtual scene data.

[0226] In some embodiments, the data acquisition module 4551 is used to acquire the generation time of each sub-data in the virtual scene data; determine the time difference between the generation time of each sub-data and the current time; use each time difference as the sub-narration delay duration of each sub-data; and combine the sub-narration delay duration of each sub-data into the narration delay duration of the virtual scene data.

[0227] In some embodiments, the data acquisition module 4551 is used to query a first mapping relationship table based on multiple types of sub-data to obtain feature labels corresponding to the multiple types of sub-data respectively, wherein the first mapping relationship table includes: the mapping relationship between each feature label and each type of sub-data.

[0228] In some embodiments, the data acquisition module 4551 is used to perform classification processing based on multiple sub-data in the virtual scene by calling a first classification model before acquiring virtual scene data, to obtain feature labels corresponding to each type of sub-data; and to construct a first mapping relationship table between each type of sub-data and its corresponding feature labels.

[0229] In some embodiments, the data acquisition module 4551 is used to call a first classification model to perform the following processing based on multiple sub-data in a virtual scene: perform feature extraction processing on the multiple sub-data to obtain multiple sub-virtual scene features; determine the type of each sub-data based on the similarity between the multiple sub-virtual scene features; and perform feature classification on the multiple sub-virtual scene features to obtain the feature labels corresponding to each type of sub-data.

[0230] In some embodiments, the data acquisition module 4551 is configured to acquire a first sample set before calling a first classification model to perform classification processing based on multiple sub-data in a virtual scene to obtain the feature labels corresponding to each type of sub-data, wherein the first sample set includes: multiple sample sub-data, the actual type corresponding to each sample sub-data, and the actual feature label corresponding to each sample sub-data; call the initialized first classification model to perform classification processing based on the first sample set to obtain the predicted type corresponding to each sample sub-data and the predicted feature label corresponding to each sample sub-data; determine a first loss function of the first classification model based on a first difference between the predicted type and the actual type and a second difference between the predicted feature label and the actual feature label; and adjust the parameters of the first classification model based on the first loss function to obtain the trained first classification model.

[0231] In some embodiments, the data acquisition module 4551 is configured to, before acquiring virtual scene data, call a second classification model based on each feature label to perform the following processing: perform probability prediction processing on the feature label to obtain the predicted probability of the feature label for different first time intervals; in response to the number of first time intervals satisfying the classification conditions being one, use the first time interval satisfying the classification conditions as the second time interval of the feature label, wherein the classification conditions include: the predicted probability corresponding to the first time interval is greater than a probability threshold; in response to the number of first time intervals satisfying the classification conditions being multiple, determine the earliest time and the latest time among the multiple first time intervals satisfying the classification conditions, and use the time interval between the earliest time and the latest time as the second time interval of the feature label; in response to the absence of a first time interval satisfying the classification conditions, determine that when the interpretation delay time is any value, the sub-data corresponding to the feature label are all data to be interpreted.

[0232] In some embodiments, the data acquisition module 4551 is used to acquire a second sample set, wherein the second sample set includes: sample feature labels corresponding to multiple sample sub-data and actual second time intervals corresponding to each sample feature label; classifying the second sample set by calling the initialized second classification model to obtain the predicted second time intervals corresponding to the sample feature labels of each sample sub-data; determining the second loss function of the second classification model based on the third difference between the predicted second time interval and the actual second time interval; and adjusting the parameters of the second classification model based on the second loss function to obtain the trained second classification model.

[0233] In some embodiments, the narration module 4553 is used to call a neural network model to perform text generation processing based on the data to be narrated, to obtain narration text; to call a neural network model to perform speech generation processing based on the narration text, to obtain narration speech; and to use the narration speech as the narration content of the virtual scene.

[0234] In some embodiments, the narration module 4553 is further configured to display a virtual scene in a human-computer interaction interface and play narration content corresponding to the virtual scene, wherein the narration content is generated based on the data processing method of the virtual scene in the embodiments of this application.

[0235] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer program or computer-executable instructions from the computer-readable storage medium and executes the computer program or computer-executable instructions, causing the electronic device to perform the virtual scene data processing method described above in this application.

[0236] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the data processing method for a virtual scene provided in this application. For example, ... Figure 3A The data processing method for the virtual scene is shown.

[0237] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0238] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0239] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0240] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0241] In summary, by filtering sub-data based on the narration delay duration and feature tags of each sub-data in the virtual scene according to the embodiments of this application, compared with the full narration scheme in related technologies, the data to be narrated in the virtual scene is simplified, resulting in a shorter delay between the narration content generated based on the data to be narrated and the current virtual scene screen, improving the smoothness of the virtual scene narration and saving the computing resources consumed in generating the virtual scene narration content. Compared with the related technologies that directly drop frames of virtual scene data based on the narration delay duration, the method of determining the data to be narrated based on the feature tags of the sub-data and the narration delay duration avoids the loss of important content in the narration and improves the user's experience of listening to the virtual scene narration.

[0242] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A data processing method for a virtual scene, characterized in that, The method includes: Acquire virtual scene data, wherein the virtual scene data includes multiple types of sub-data; Obtain the narration delay duration of the virtual scene data and the feature labels corresponding to the multiple types of sub-data; Based on the feature labels of each type of sub-data and the interpretation delay duration, at least one type of sub-data is filtered out from the multiple types of sub-data to obtain the data to be interpreted; The narration content for the virtual scene is generated based on the data to be narrated.

2. The method according to claim 1, characterized in that, The explanation delay duration includes: the sub-explanation delay duration corresponding to each of the multiple types of sub-data; The step of filtering out at least one type of sub-data from the multiple types of sub-data based on the feature labels of each type of sub-data and the interpretation delay duration to obtain the data to be interpreted includes: From the multiple preset first time intervals for each of the feature labels, determine the target first time interval in which the sub-interpretation delay time of each of the sub-data is located; Obtain the second time interval corresponding to each feature label, wherein the second time interval of the feature label includes a portion of the first time interval corresponding to the feature label; Based on the second time interval of each feature label and the first time interval of the target, sub-data of the target type is determined from the multiple types of sub-data, and the sub-data of the target type is filtered out from the multiple types of sub-data to obtain the data to be explained.

3. The method according to claim 2, characterized in that, The step of determining the target first time interval in which the sub-interpretation delay of each sub-data belongs from a plurality of preset first time intervals for each feature label includes: Obtain a decision tree model, wherein the decision tree model includes multiple first root nodes and a decision path corresponding to each first root node, the first root node represents the feature label of the sub-data, and each decision path includes multiple first leaf nodes, the first leaf nodes representing a first time interval corresponding to the feature label of each sub-data. For each type of sub-data, traverse the decision paths corresponding to the sub-data of that type in the decision tree model; In response to the fact that the sub-interpretation delay duration of the sub-data of the aforementioned type is within the first duration interval of the traversed first leaf node, the first duration interval in which the sub-interpretation delay duration is located is taken as the target first duration interval of the sub-data of the aforementioned type.

4. The method according to claim 2, characterized in that, The step of obtaining the second time interval corresponding to each feature label includes: Obtain a computational tree model, wherein the plurality of second leaf nodes of the computational tree model include a second time interval associated with each of the sub-data; The process involves determining target type sub-data from the plurality of types of sub-data based on the second time interval of each feature label and the target first time interval, and then filtering out the target type sub-data from the plurality of types of sub-data to obtain the data to be interpreted, including: The decision paths corresponding to the second root node of the computation tree model are traversed one by one. The second root node represents the feature label of the sub-data. Each decision path includes multiple second leaf nodes. The multiple second leaf nodes in the same decision path are sorted in ascending order according to the number of first time intervals included in the corresponding second time interval. In response to the second leaf node being traversed satisfying the filtering condition, the following sub-data are filtered out: sub-data corresponding to the target second leaf node that satisfies the filtering condition, and sub-data of each subsequent second leaf node connected to the target second leaf node, wherein the filtering condition includes: the target first time interval of the feature label is located in the second time interval corresponding to the feature label; In response to the end of the traversal process, the remaining data in the multiple types of sub-data are taken as the data to be explained.

5. The method according to claim 2, characterized in that, The step of obtaining the second time interval corresponding to each feature label includes: Based on each feature label, a second mapping relationship table is queried to obtain the second time interval corresponding to each feature label. The second mapping relationship table includes the mapping relationship between each feature label and the corresponding second time interval. The process involves determining target type sub-data from the plurality of types of sub-data based on the second time interval of each feature label and the target first time interval, and then filtering out the target type sub-data from the plurality of types of sub-data to obtain the data to be interpreted, including: Iterate through each of the aforementioned feature labels; In response to the fact that the target first time interval of the traversed feature label is located in the second time interval corresponding to the feature label, it is determined that the type of the sub-data corresponding to the traversed feature label is the target type, and the sub-data of the target type is filtered out; In response to the end of the traversal process, the remaining data in the multiple types of sub-data are taken as the data to be explained.

6. The method according to claim 2, characterized in that, Before acquiring the virtual scene data, the method further includes: Based on each of the aforementioned feature labels, the second classification model is invoked to perform the following processing: The feature labels are subjected to probability prediction processing to obtain the predicted probabilities of the feature labels for different first time intervals; In response to the fact that there is only one first time interval that satisfies the classification condition, the first time interval that satisfies the classification condition is used as the second time interval of the feature label, wherein the classification condition includes: the predicted probability corresponding to the first time interval is greater than the probability threshold; In response to the fact that there are multiple first time intervals that satisfy the classification conditions, the earliest time and the latest time among the multiple first time intervals that satisfy the classification conditions are determined, and the time interval between the earliest time and the latest time is taken as the second time interval of the feature label. In response to the absence of a first time interval that satisfies the classification conditions, it is determined that when the explanation delay time is any value, the sub-data corresponding to the feature label are all data to be explained.

7. The method according to claim 6, characterized in that, The method further includes: Obtain a second sample set, wherein the second sample set includes: sample feature labels corresponding to multiple sample sub-data and the actual second time interval corresponding to each sample feature label; Based on the second sample set, the initialized second classification model is called to perform classification processing to obtain the predicted second time interval corresponding to the sample feature label of each sample sub-data. Based on the third difference between the predicted second time interval and the actual second time interval, the second loss function of the second classification model is determined. The parameters of the second classification model are adjusted based on the second loss function to obtain the trained second classification model.

8. The method according to claim 1, characterized in that, The acquisition of virtual scene data includes: Obtain the virtual scene features of the current virtual scene; Each narration event in the virtual scene is identified based on the virtual scene features, wherein the narration event is the content in the virtual scene that needs to be narrated; The data corresponding to each of the aforementioned narration events is combined into the virtual scene data.

9. The method according to claim 1, characterized in that, The narration delay for acquiring the virtual scene data includes: Obtain the generation time of each sub-data in the virtual scene data; Determine the time difference between the generation time of each of the sub-data items and the current time; Each time difference is used as the sub-interpretation delay duration for each sub-data. The sub-narration delay duration of each of the sub-data is combined to form the narration delay duration of the virtual scene data.

10. The method according to claim 1, characterized in that, Obtaining the feature labels corresponding to the multiple types of sub-data respectively, including: Based on the multiple types of sub-data, a first mapping relationship table is queried to obtain the feature labels corresponding to the multiple types of sub-data respectively. The first mapping relationship table includes the mapping relationship between each feature label and each type of sub-data.

11. The method according to claim 10, characterized in that, Before acquiring the virtual scene data, the method further includes: Based on multiple sub-data in the virtual scene, the first classification model is called to perform classification processing to obtain the feature label corresponding to each type of sub-data; Construct the first mapping table between each type of sub-data and its corresponding feature label.

12. The method according to claim 11, characterized in that, The process of classifying multiple sub-data points in the virtual scene using a first classification model to obtain feature labels corresponding to each type of sub-data includes: Based on multiple sub-data points in the virtual scene, the first classification model is invoked to perform the following processing: Feature extraction processing is performed on the multiple sub-data to obtain multiple sub-virtual scene features; The type of each sub-data is determined based on the similarity among the features of the multiple sub-virtual scenes; The features of the multiple sub-virtual scenes are classified to obtain feature labels corresponding to each type of sub-data.

13. The method according to claim 11, characterized in that, Before performing classification processing based on multiple sub-data in the virtual scene using a first classification model to obtain the feature label corresponding to each type of sub-data, the method further includes: Obtain a first sample set, wherein the first sample set includes: multiple sample sub-data, the actual type corresponding to each sample sub-data, and the actual feature label corresponding to each sample sub-data; Based on the first sample set, the initialized first classification model is called to perform classification processing to obtain the prediction type corresponding to each sample sub-data and the prediction feature label corresponding to each sample sub-data; Based on the first difference between the predicted type and the actual type, and the second difference between the predicted feature label and the actual feature label, the first loss function of the first classification model is determined; The parameters of the first classification model are adjusted based on the first loss function to obtain the trained first classification model.

14. The method according to any one of claims 1 to 13, characterized in that, The process of generating narration content for the virtual scene based on the data to be narrated includes: Based on the data to be explained, a neural network model is invoked to perform text generation processing to obtain the explanatory text; Based on the narration text, the neural network model is invoked to perform speech generation processing to obtain the narration speech; The narration is used as the narration content for the virtual scene.

15. A data processing method for a virtual scene, characterized in that, The method includes: Display virtual scenes in the human-computer interaction interface; Play the narration content corresponding to the virtual scene, wherein the narration content is generated based on the data processing method of the virtual scene according to any one of claims 1 to 14.

16. A data processing device for a virtual scene, characterized in that, The device includes: A data acquisition module is used to acquire virtual scene data, wherein the virtual scene data includes multiple types of sub-data; The data acquisition module is also used to acquire the narration delay duration of the virtual scene data and the feature labels corresponding to the multiple types of sub-data respectively; The data pruning module is used to filter out at least one type of sub-data from the multiple types of sub-data based on the feature labels of each type of sub-data and the interpretation delay duration, so as to obtain the data to be interpreted; The narration module is used to generate narration content for the virtual scene based on the data to be narrated.

17. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data processing method for the virtual scene according to any one of claims 1 to 15.

18. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the data processing method for the virtual scene as described in any one of claims 1 to 15 is implemented.

19. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the data processing method for the virtual scene according to any one of claims 1 to 15 is implemented.

Images