Virtual object processing method, device and equipment, computer program product and computer readable storage medium

Abstract

The invention provides a virtual object processing method and device, equipment, a computer program product and a computer readable storage medium. The method comprises the following steps: acquiring text description information of an image of a virtual scene at a current time step; obtaining semantic understanding information of the virtual scene in the previous time step; encoding the text description information and the semantic comprehension information of the previous time step to obtain the semantic comprehension information of the virtual scene at the current time step; acquiring an action parameter of the virtual object in the previous time step, and acquiring feature information of the virtual object in the current time step; action prediction processing is carried out through the semantic understanding information of the current time step, the feature information of the current time step and the action parameters of the previous time step, the action parameters of the virtual object in the current time step are obtained, and the action parameters of the current time step are used for driving the virtual object to execute actions in the current time step. According to the method and the device, the virtual object can be intelligently and automatically controlled.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, computer program product, and computer-readable storage medium for processing virtual objects. Background Art

[0002] Most virtual object automation solutions in related technologies are based on user interface (UI) recognition and image recognition (e.g., MeanShift, frame difference, etc.). UI-based recognition typically involves identifying UI controls within an operating system (e.g., Android). However, when faced with complex scenarios, general automation solutions often struggle to accurately control the movements of virtual objects. Summary of the Invention

[0003] Embodiments of the present application provide a method, apparatus, device, computer program product, and computer-readable storage medium for processing a virtual object, which can intelligently and automatically control the virtual object.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present invention provides a method for processing a virtual object, the method comprising:

[0006] Obtaining text description information of an image of a virtual scene at a current time step, wherein the image includes at least one virtual object;

[0007] Obtaining semantic understanding information of the virtual scene at a previous time step;

[0008] Obtaining semantic understanding information of the virtual scene at the current time step by encoding the text description information and the semantic understanding information of the previous time step;

[0009] Obtaining the action parameters of the virtual object at the previous time step, and obtaining the feature information of the virtual object at the current time step;

[0010] Action prediction processing is performed through the semantic understanding information of the current time step, the feature information at the current time step and the action parameters of the previous time step to obtain the action parameters of the virtual object at the current time step, wherein the action parameters of the current time step are used to drive the virtual object to perform action at the current time step.

[0011] An embodiment of the present application provides a device for processing a virtual object, the device comprising:

[0012] a text information generation module, configured to obtain text description information of an image of a virtual scene at a current time step, wherein the image includes at least one virtual object;

[0013] A contextual semantic understanding module, used to obtain semantic understanding information of the virtual scene at the previous time step;

[0014] The context semantic understanding module is further configured to encode the text description information and the semantic understanding information of the previous time step to obtain the semantic understanding information of the virtual scene at the current time step;

[0015] An action generation module, configured to obtain the action parameters of the virtual object at the previous time step and obtain the feature information of the virtual object at the current time step;

[0016] The action generation module is also used to perform action prediction processing based on the semantic understanding information of the current time step, the feature information at the current time step and the action parameters of the previous time step to obtain the action parameters of the virtual object at the current time step, wherein the action parameters of the current time step are used to drive the virtual object to perform action at the current time step.

[0017] An embodiment of the present application provides an electronic device, comprising:

[0018] a memory for storing computer-executable instructions;

[0019] The processor is configured to implement the virtual object processing method provided in the embodiment of the present application when executing the computer executable instructions stored in the memory.

[0020] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing a virtual object processing method provided in an embodiment of the present application when executed by a processor.

[0021] An embodiment of the present application provides a computer program product, including a computer program or computer-executable instructions. When the computer program or computer-executable instructions are executed by a processor, the virtual object processing method provided in the embodiment of the present application is implemented.

[0022] The embodiments of the present application have the following beneficial effects:

[0023] Based on the action parameters of the virtual object in the previous time step, it is possible to fully utilize various information of the current time step of the virtual scene (including the semantic understanding information of the image of the virtual scene in the current time step and the feature information of the virtual object in the current time step) to predict action, so that the predicted action parameters of the current time step can match the context of the scene and the characteristics of the virtual scene itself. Therefore, when facing various complex virtual scenes, the action parameter prediction of the virtual object can be realized in units of time steps, ensuring the continuity and accuracy of the action of the virtual object. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a schematic diagram of the structure of the virtual object processing system architecture provided by an embodiment of the present application;

[0025] Figure 2 is a structural diagram of an electronic device provided in an embodiment of the present application;

[0026] Figure 3A Schematic diagram of the architecture of a model for processing virtual objects provided by an embodiment of the present application;

[0027] Figure 3B This is a schematic diagram of the structure of the large language model provided in the embodiment of the present application;

[0028] Figure 3C Schematic diagram of the structure of the recurrent neural network provided in the embodiment of the present application;

[0029] Figure 3D is a schematic diagram of an optional structure of an action model provided in an embodiment of the present application;

[0030] Figure 4A This is a first flow chart of the method for processing virtual objects provided by an embodiment of the present application;

[0031] Figure 4B This is a second flow chart of the method for processing virtual objects provided in an embodiment of the present application;

[0032] Figure 4C This is a third flow chart of the method for processing virtual objects provided in an embodiment of the present application;

[0033] Figure 4D This is a fourth flow chart of the method for processing a virtual object provided in an embodiment of the present application;

[0034] Figure 4E This is a fifth flow chart of the method for processing a virtual object provided in an embodiment of the present application;

[0035] Figure 4F This is a sixth flow chart of the method for processing a virtual object provided in an embodiment of the present application;

[0036] Figure 4G This is a seventh flow chart of the method for processing a virtual object provided in an embodiment of the present application;

[0037] Figure 4H This is an eighth flow chart of the method for processing virtual objects provided in an embodiment of the present application;

[0038] Figure 4I This is a ninth flow chart of a method for processing a virtual object provided in an embodiment of the present application;

[0039] Figure 5A This is a first schematic diagram of the construction principle of the training sample provided in the embodiment of the present application;

[0040] Figure 5B This is a second schematic diagram of the construction principle of the training sample provided in the embodiment of the present application;

[0041] Figure 6 Schematic diagram of a game automation framework provided by an embodiment of the present application;

[0042] Figure 7 This is a schematic diagram of the interactive process of game automation provided by an embodiment of the present application;

[0043] Figure 8 is a schematic diagram of a game image provided in an embodiment of the present application;

[0044] Figure 9A This is a schematic diagram of the target recognition framework provided by the embodiment of the present application;

[0045] Figure 9B Schematic diagram of the posture estimation framework provided by the embodiment of the present application;

[0046] Figure 10 It is a schematic diagram of the convolution processing of an image provided in an embodiment of the present application.

[0047] It should be pointed out that the above-mentioned "first" and "second" are only used to distinguish different solutions, and do not represent the degree of distinction between the advantages and disadvantages of the solutions or the priority in the implementation process. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0049] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0050] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0051] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0052] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0053] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0054] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0055] 1) Large Language Model (LLM) is a machine learning model designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, and more. Large language models are characterized by their large scale and billions of parameters, which help them learn complex patterns in language data. They are usually based on deep learning architectures (such as ChatGPT, PaLM4, etc.). Large language models refer to deep learning models trained using large amounts of text data, containing billions or even more parameters. They can be used to generate natural language text and understand the meaning of natural language text. Through training, the model can learn the statistical laws and semantic associations of language to build a huge language knowledge base, thereby simulating human language understanding and generation capabilities. Large language models have the following characteristics:

[0056] Learning ability: Through training on massive amounts of text data, large language models can learn rich language knowledge and expressions, including grammar, semantics, and common expression habits.

[0057] Pattern recognition: Large language models can identify common text patterns and semantic associations, such as the co-occurrence relationship between words, the logical structure and semantic roles of sentences, etc.

[0058] Contextual understanding: Large language models can capture contextual information in text, understand the impact of previous text on subsequent text, and generate appropriate responses based on the context.

[0059] Generative capabilities: Large language models can generate relevant natural language text based on input information, including answering questions, generating articles, and conducting conversations.

[0060] Resolving ambiguity: Despite the polysemy and ambiguity of language, large language models resolve ambiguity through contextual information and language regularities, providing more accurate and appropriate text generation or understanding.

[0061] Large language models have a wide range of application scenarios. They can be applied to intelligent customer service, intelligent question and answer, natural language generation, advertising recommendations, games and other fields. They can improve the efficiency and accuracy of human-computer interaction and enhance user experience.

[0062] 2) Recurrent Neural Network (RNN) is a neural network that can process sequence data. Unlike traditional feedforward neural networks, RNN can use the output of the previous time step as the input of the current time step, thereby modeling sequence data.

[0063] 3) Virtual objects refer to the target objects when performing automated operations on objects in the image of the virtual environment. Taking the game scene as an example, the virtual objects can be the game characters, non-player characters (NPCs), and game operation controls (such as controls for releasing skills, game setting controls, game start controls, game login controls, etc.) of the player in the current image.

[0064] 4) Text description information is a semantic description of the virtual objects contained in the image of the virtual environment. Taking the game scene as an example, the text description information can include the name, type, coordinate information of the game character in the current image, coordinate information of the game operation controls (such as controls for releasing skills, game setting controls, start game controls, etc.), etc.

[0065] 5) Semantic understanding information is the semantic information obtained after modeling the temporal context based on the text description information. It can be represented as a vector (such as the hidden state output by the hidden layer in the RNN) or as text (such as converting the hidden state output by the RNN into a character index stream, thereby obtaining the corresponding text based on the character index).

[0066] 6) Feature information refers to the basic attributes of a virtual object and at least one dynamic attribute at the current time step. Taking a game character as an example, the feature information may include the identity identifier (such as ID, etc.) of the current game character (virtual object), professional attributes (including attribute bonuses obtained by the game character through game skins, such as increasing the game character's movement speed by wearing game skins, etc.), remaining health, the number of currently equipped ammunition, etc.

[0067] 7) Memory Network is a model for structured storage and retrieval of information that can help neural networks better handle long-term dependencies and utilize historical information. In implementation, memory networks can be trained in conjunction with the feedforward and feedback propagation of neural networks to achieve the storage and utilization of historical information. By continuously reading and writing memory, the model can gradually accumulate an understanding of historical information and effectively retrieve and utilize this information when needed.

[0068] 8) Connection relationship refers to the definition and modeling of the associations between different key points in posture estimation. These associations can be used to describe the posture structure and movement of the human body or object. For example, a skeletal model is used to represent the posture structure of the human body, where the connection relationship between key points corresponds to the connection of bones. The key points of the human body may include the head, neck, shoulders, arms, legs, etc., and the connection relationship can be defined as the head connected to the neck, the neck connected to the shoulders, the shoulders connected to the arms, and so on. The connection relationship can be represented by a graph model. In the graph model, each key point is represented as a node in the graph, and the connection relationship is represented as the edge between the nodes. By defining the connection relationship between nodes, a graph representing the posture structure can be established. In the graph, the edges between nodes can have different properties, such as length, angle, etc., which are used to describe the relative position and constraint relationship between key points.

[0069] 9) Time step refers to the observation point in the sequence data. The time interval between one observation point and the next observation point controls the sampling frequency and time scale of the time series data. For example, if data is collected once a day, then each time step represents one day; if data is recorded once an hour, then each time step is one hour.

[0070] 10) Execution action refers to the action or behavior of a virtual object in a virtual environment. Taking a game scene as an example, execution actions may include the movement, escape, jump, attack, etc. of a game character.

[0071] 11) Execution status refers to the status of virtual objects in the virtual environment. Taking the game scene as an example, the execution status may include skill releasing, skill releasing completed, skill cooling down, character resurrection, etc.

[0072] 12) Control strategy refers to the guiding principles for operating within the virtual environment to achieve game objectives. Control strategy encompasses multiple aspects, such as character selection, skill pairing, equipment configuration, and team coordination. In short, control strategy in games involves multiple aspects, and players need to flexibly adjust their strategies based on the game type, character characteristics, and battlefield environment. Only through continuous learning and reflection in practice can one achieve better results in the game. The following describes each aspect of control strategy:

[0073] Character selection: Choose the appropriate character based on the game type and gameplay. For example, in a MOBA game, choose a hero character with synergy to leverage your strengths in team battles. In a role-playing game, choose the character's gender, class, and skill development based on your personal preferences and gaming needs.

[0074] Skill Combination: Properly combine and use skills to improve the combat effectiveness of the game character. This requires players to understand the attributes, effects, and applicable scenarios of each skill in order to maximize the effectiveness of the skills in battle.

[0075] Equipment Configuration: Choose the appropriate equipment based on the characteristics and needs of your character. Equipment can improve your character's attributes, attack power, defense, etc., so proper equipment configuration is crucial to victory in the game.

[0076] Teamwork: In multiplayer online games, teamwork is key to victory. Players need to understand their teammates' roles, skills, and tactics to coordinate their efforts in combat. Effective communication and coordination are also crucial elements of teamwork.

[0077] Tactical Application: In games, develop appropriate tactics based on the strength of both sides and the battlefield environment. For example, in strategic games, rationally allocate troops, control resources, and develop expansion strategies. In competitive games, study the opponent's tactics and weaknesses to gain an advantage in the game.

[0078] Mental health: Maintaining a positive attitude and proactively responding to challenges and pressures in the game. Mental health includes self-discipline, patience, and calmness, qualities that help players make wise decisions in the game.

[0079] Continuous learning and summarizing: Continuously learn new knowledge, skills and strategies of the game, summarize your own experiences and lessons in the game, so as to continuously improve your gaming level.

[0080] State management (when the game character's health value drops to a preset value, reduce the game character's movement speed or attack power, etc.) Close pop-up ads before or during the game.

[0081] The virtual object automation solution based on UI recognition in the related technology refers to simulating user operations and interactions on the user interface by writing and executing test scripts. However, since most virtual environments use development engines such as UE or Unity, UI controls may not have clear attributes or identifications and are subject to interference such as occlusion, animation, and pop-ups, so there is a problem that controls cannot be recognized; the virtual object automation solution based on image recognition in the related technology refers to identifying specific targets (virtual objects) in the image of the virtual environment, and then performing automated actions according to the process of the virtual environment. However, this method is susceptible to problems such as multi-target interference, which makes it difficult for automation to complete the original goal, especially in complex scenarios, such as account creation, novice tutorials, and other scenarios in games, as well as sudden situations such as unexpected pop-ups. Therefore, the automation solution of the related technology is difficult to perform intelligent and automated judgment on virtual objects.

[0082] In order to solve the above problems, embodiments of the present application provide a method, apparatus, device, computer program product, and computer-readable storage medium for processing virtual objects, which can intelligently and automatically control virtual objects.

[0083] The following describes exemplary applications of the device provided in the embodiments of the present application. The electronic device provided in the embodiments of the present application can be implemented as various types of terminals such as laptops, tablet computers, desktop computers, set-top boxes, smart phones, smart speakers, smart watches, smart TVs, and car terminals, and can also be implemented as servers.

[0084] See also Figure 1 , Figure 1 This is a schematic diagram of the structure of the virtual object processing system architecture provided by the embodiment of the present application, for example, Figure 1 The server 100, the terminal device 200 and the network 300 are involved. The terminal device 200 is connected to the server 100 via the network 300, wherein the network 300 can be a wide area network or a local area network, or a combination of the two.

[0085] In some embodiments, the embodiments of the present application can be implemented collaboratively by a server and a terminal device. For example, the terminal device 200 sends an image of the virtual environment at the current time step to the server 100. The server 100 obtains the action parameters of the virtual object included in the image at the current time step through the virtual object processing method provided in the embodiments of the present application, and sends the action parameters to the terminal device 200. The terminal device 200 then performs the relevant action based on the action parameters of the virtual object at the current time step.

[0086] In other embodiments, the present invention can be implemented by a terminal device alone. The terminal device 200 sends a virtual object automation request to the server. The server 100 receives the virtual object automation request and sends the model for processing the virtual object provided by the present invention to the terminal device 200. Figure 3A , Figure 3A This is a schematic diagram of the architecture of a model for processing virtual objects provided in an embodiment of the present application. The terminal device 200 receives the model sent by the server and downloads it locally. Through the image of the model and the virtual environment at the current time step (for example, the game image of an offline version of a stand-alone game), the action parameters of the virtual object included in the image at the current time step are obtained. The terminal device 200 performs relevant actions on the virtual object through the action parameters (for example, the virtual object controlled by the action parameters obtained by the model in the terminal device interacts with the virtual object manually controlled by the user).

[0087] In some embodiments, the terminal device or server can implement the processing method of the virtual object provided in the embodiment of the present application by running various computer executable instructions or computer programs. For example, the computer executable instructions can be microprogram-level commands, machine instructions or software instructions. The computer program can be a native program or software module in the operating system. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form. The terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc.

[0088] In some embodiments, multiple servers may form a blockchain network, with server 100 being a node on the blockchain network. Information connections may exist between each node in the blockchain network, and information may be transmitted between nodes via the aforementioned information connections. Data related to the virtual object processing method provided in the embodiments of the present application may be stored on the blockchain.

[0089] In some embodiments, the server 100 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal and the server may be connected directly or indirectly via wired or wireless communication, which is not limited in the embodiments of the present application.

[0090] The embodiments of the present application can be implemented with the help of artificial intelligence (AI) technology. Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to achieve the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0091] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, pre-trained models, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI disciplines. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0092] See also Figure 2 , Figure 2 is a structural diagram of an electronic device provided in an embodiment of the present application, Figure 2 The electronic device 400 shown can be the server 100 or the terminal device 200. Figure 2 The electronic device 400 shown includes: at least one processor 110, a memory 130 and at least one network interface 120. The various components in the encryption server 100 are coupled together via a bus system 140. It is understood that the bus system 140 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 140 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 140 is not described in detail. Figure 2 Various buses are labeled as bus system 140 .

[0093] The processor 110 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0094] The memory 130 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, a hard drive, an optical drive, etc. The memory 130 may optionally include one or more storage devices that are physically remote from the processor 110.

[0095] The memory 130 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 130 described in the embodiments of the present application is intended to include any suitable type of memory.

[0096] In some embodiments, the memory 130 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0097] Operating system 131, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0098] A network communication module 132 for reaching other electronic devices via one or more (wired or wireless) network interfaces 120 , exemplary network interfaces 120 including Bluetooth, Wi-Fi, and Universal Serial Bus (USB);

[0099] In some embodiments, the apparatus provided in the embodiments of the present application may be implemented in software. Figure 2 A virtual object processing device 133 stored in memory 130 is shown. This device can be software in the form of a program or plug-in, and includes the following software modules: a text information generation module 1331, a contextual semantic understanding module 1332, and an action generation module 1333. These modules are logical and can be arbitrarily combined or further separated based on the functions they implement. The functions of each module will be described below.

[0100] In other embodiments, the apparatus provided in the embodiments of the present application may be implemented in hardware. As an example, the apparatus provided in the embodiments of the present application may be a processor in the form of a hardware decoding processor, which is programmed to execute the virtual object processing method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor may be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0101] See also Figure 3A , Figure 3A The model shown includes a large language model, a recurrent neural network model and an action model, wherein the large language model is used to perform semantic understanding of the image of the virtual scene at the current time step and output text description information; the recurrent neural network is used to perform time modeling based on the text description information obtained by the large language model to obtain semantic understanding information of the context; and the action model is used to obtain the action parameters of the virtual object at the current time step based on the obtained semantic understanding information.

[0102] See also Figure 3B , Figure 3B This is a schematic diagram of the structure of the large language model provided in the embodiment of the present application. The large language model includes an encoding module and a decoding module (such as Figure 3B The stacked structure of multiple encoding modules and multiple decoding modules is shown in Figure 2. Each module contains a multi-head attention layer and a feedforward neural network. The attention mechanism is a basic component of large language models; see Figure 3C , Figure 3C This is a schematic diagram of the structure of the recurrent neural network provided by the embodiment of the present application. Taking the recursion three times as an example, the output of the hidden layer at the previous time step and the input of the input layer at the current time step are used together as the input of the hidden layer at the current time step, so that the recurrent neural network model can generalize sequence data of different lengths, thereby learning the temporal information and semantic information in the input data; see Figure 3D , Figure 3DThis is a schematic diagram of an optional structure of the action model provided in an embodiment of the present application. Taking the input training sample as the text description information (which can also be semantic understanding information) of the image of the virtual scene at the current time step, the feature information at the current time step, and the action parameters of the historical time step as an example, the text information is processed by word embedding in the input layer to obtain a text vector representation of the text information, and the text vector representation of the text information can be obtained in the middle layer through a time sequence network (such as RNN, long short-term memory neural network (Long Short-term MemoryNetworks, LSTM), Transformer, etc.) gradually read the text vector representation and convert the text vector representation into a hidden state representation. The input of the intermediate layer includes the information of the action parameters of the historical time steps in the memory network. Finally, the hidden state representation is mapped to the action parameters of the current time step through the output layer (such as the fully connected layer). At the same time, the action parameters are stored in the memory network, which facilitates the long-term memory generation of the action model and improves the generalization ability of the action model. Here, the memory network includes a memory matrix. Each row of the memory matrix represents a memory unit. Each memory unit includes historical information (such as action parameters in previous time steps). The corresponding action parameters are retrieved and read in the memory matrix (such as calculating the similarity between the query vector and the memory unit, and obtaining the reading result according to the weighted similarity) and the text vector representation as the input of the intermediate layer.

[0103] The following will describe the data encryption method provided by the embodiment of the present application in conjunction with the exemplary application and implementation of the server provided by the embodiment of the present application. Figure 4A , Figure 4A This is a first flow chart of the method for processing virtual objects provided by the embodiment of the present application, which will be combined with Figure 4A The steps shown are explained.

[0104] In step 101, text description information of an image of a virtual scene at a current time step is obtained, wherein the image includes at least one virtual object.

[0105] In some embodiments, text description information of an image of a virtual scene at a current time step is obtained, wherein the interval between adjacent time steps can be customized, for example, the timestamp in each frame of the image can be used as an execution time step, and the image includes at least one virtual object that performs an action, wherein the virtual object is an object that can be manipulated in the virtual scene. Taking a game scene as an example, the virtual objects include game characters, non-character players, props, buttons, triggers, and other elements that can manipulate and affect the game state.

[0106] In some embodiments, see Figure 4B , Figure 4A The step 101 shown can be implemented by following steps 1011 to 1015, which are described in detail below.

[0107] In step 1011, target recognition is performed on the image of the virtual scene at the current time step to obtain a target recognition result, wherein the target recognition result includes the category of the virtual object in the image and the position of the bounding box of the virtual object.

[0108] In some embodiments, for example, if the virtual object contained in the image is a vehicle, the target recognition result may include the vehicle category (car, truck, etc.) and the coordinate position of the vehicle's bounding box (such as the coordinate positions corresponding to the four corners of the rectangular bounding box).

[0109] In some embodiments, see Figure 4C , Figure 4B Step 1011 shown can be implemented by following steps 10111 to 10113, which are described in detail below.

[0110] In some embodiments, see Figure 9A , Figure 9A This is a schematic diagram of the target recognition framework provided by the embodiment of the present application. Taking the gender of the game character in the classification image as an example, after the image is input, the feature information of the image is obtained through feature extraction operation (such as feature extraction through the Darknet network), and finally the feature information of the image is mapped through the two branches of classification and regression, such as Figure 9A As shown, the image features are mapped to the probability value of each category through the classification task (the probability of being male is 85%, and the probability of being female is 45%), and the coordinate position of the bounding box is obtained through the regression task (corresponding to Figure 9A (100,24,243,80), where (100,24) is the coordinate value of the upper left corner and (243,80) is the coordinate value of the lower right corner). Finally, the category and bounding box of the target are displayed in the original image to obtain the output image result of the target recognition (corresponding to Figure 9A In the bounding box regression task, the network generates multiple prediction boxes for each target to be identified. Non-maximum suppression (NMS) can be used to remove redundant prediction boxes and obtain the most representative bounding box.

[0111] In step 10111, convolution processing is performed on the image of the virtual scene at the current time step to obtain a feature representation of the image.

[0112] In some embodiments, a preset convolution kernel (e.g., a convolution kernel of size 3×3) is used to slide on the image according to a preset step size. After each slide, the grayscale value of the pixel on the image point is multiplied by the value on the corresponding convolution kernel. Then, all the multiplied values ​​are added together as the grayscale value of the pixel on the image corresponding to the middle pixel of the convolution kernel. After the slide is completed, a feature map composed of multiple grayscale value results is used as the feature representation of the image. See Figure 10 , Figure 10 is a schematic diagram of the convolution processing of an image provided in an embodiment of the present application, Figure 10 An exemplary convolution operation process is shown when the convolution kernel size is 3×3.

[0113] In step 10112, classification regression processing is performed through feature representation to obtain the category of each virtual object and the position of the bounding box.

[0114] In some embodiments, converting the obtained feature representation of the image into the category of the virtual object (target) and the position of the bounding box involves two main tasks: classification and regression. For the classification task, the feature representation of the image can be mapped to the probability value of each category of the current target through a fully connected layer and an activation function (such as a Softmax activation function), and the category with the largest probability value is used as the output of the virtual object category; for the regression task, coordinate regression can be performed through another fully connected layer to output the four coordinate values ​​of the bounding box.

[0115] In some embodiments, the category of each virtual object and the location of the bounding box may be obtained through a deep learning algorithm.

[0116] For example, deep learning algorithms can use the You Only Look Once (YOLO) model. YOLO is a single-stage object recognition (object detection) model that achieves end-to-end object recognition by transforming the object recognition problem into a regression problem. YOLO divides the image into grids and predicts the bounding box of the virtual object in each grid, along with the corresponding confidence score and class probability, to achieve real-time object recognition.

[0117] In step 10113, text generation processing is performed on the category and position to obtain the target recognition result.

[0118] In some embodiments, a large language model can be used to generate text based on the category and location of the virtual object. For example, the category and location are converted into text form and combined with a specific tag (such as "[OBJ]") to form an input prefix. Finally, the feature representation of the image is integrated with the input prefix as an input sequence for the large language model. The large language model forms text description content related to the identified virtual object based on the input information (corresponding to the target recognition result).

[0119] As an example, a large language model is used to obtain a text description related to the category and bounding box of a virtual object, as shown in the following formula:

[0120] y1=f(x,θ)=GPT(x,Conv(x,W,b)) (1)

[0121] Here, x represents the input image, y1 represents the output object recognition result, and θ represents the model parameters. Conv represents a convolutional layer, W represents the convolution kernel weight matrix, and b represents the convolution kernel bias vector. GPT represents a ChatGPT-based image-to-text generation model. It can generate a suitable text description containing the category and bounding box of the virtual object based on the image and the output of the convolution layer. This achieves object recognition by leveraging the powerful language and visual capabilities of ChatGPT and the feature extraction capabilities of the convolution layer.

[0122] Continue to see Figure 4B In step 1012, semantic segmentation is performed on the image of the virtual scene at the current time step to obtain a semantic segmentation result, wherein the semantic segmentation result includes at least one region in the image and a category of the region.

[0123] In some embodiments, see Figure 4D , Figure 4B Step 1012 shown can be implemented by following steps 10121 to 10123, which are described in detail below.

[0124] In step 10121, the category label of each pixel in the image is obtained.

[0125] In some embodiments, an image can be received as input by a deep learning algorithm, and a category label corresponding to each pixel can be output, wherein the category label of each pixel can be determined by selecting the output category with the highest probability.

[0126] Taking the U-Net deep learning algorithm as an example, U-Net consists of an encoder, a decoder, and an output layer. The encoder progressively extracts feature representations of the image. By stacking convolutional and pooling layers, it gradually reduces the size of the feature map while increasing the number of channels to capture features at different levels of abstraction. The decoder maps the features extracted by the encoder back to the size of the original input image. Combining the encoder's feature information, the decoder uses deconvolution layers (also known as transposed convolutions) for upsampling. Skip connections connect the corresponding feature map from the encoder to the decoder, helping to recover detailed information. The output layer uses a convolutional layer to output a feature map of the same size as the input image. Each pixel in the feature map is assigned a predicted probability for a class label. The class corresponding to the predicted probability above a probability threshold is used as the corresponding class label for the pixel. For example, if three objects are segmented in an image: game characters, virtual vehicles, and roads, the corresponding class labels might be: game characters (labeled 1); virtual vehicles (labeled 2); and roads (labeled 3). After processing through the U-Net model, the class label for each pixel is obtained.

[0127] In step 10122, the image is segmented into regions with different category labels according to the category label of each pixel in the image, where different category labels represent different semantics.

[0128] In some embodiments, the image is segmented into different regions according to category labels, that is, the category labels of pixels in the same region are the same.

[0129] Continuing with the above example, after obtaining the category label for each pixel, we can perform clustering or connected component analysis on adjacent pixels, and group the interconnected pixels into a connected region, thereby obtaining a more continuous and complete semantic segmentation result. For example, by clustering the pixels with label 1 among the above pixels, we can obtain the complete region corresponding to the category "game character".

[0130] In step 10123, text generation processing is performed on the regions with different category labels to obtain semantic segmentation results.

[0131] In some embodiments, text generation can be performed based on different regions of an image using a large language model, as shown in the following formula:

[0132] y2=f(x,θ)=GPT(x,Label(x)) (2)

[0133] Where x represents the input image, y2 represents the output semantic segmentation result, and θ represents the model parameters. Label represents an automated labeling model (such as SegNet, U-Net, etc.), which generates a matrix of the same size as the image, with each element representing the category label of the corresponding pixel. GPT represents a ChatGPT-based image-to-text generation model, which generates a suitable text description based on the image and the annotated category labels, including the different regions and categories in the image.

[0134] Continue to see Figure 4B In step 1013, instance segmentation is performed on the image of the virtual scene at the current time step to obtain an instance segmentation result, wherein the instance segmentation result includes the category of the virtual object in the image and the mask area of ​​the virtual object.

[0135] In some embodiments, see Figure 4E , Figure 4B Step 1013 shown can be implemented by following steps 10131 to 10133, which are described in detail below.

[0136] In step 10131, target recognition is performed on the image of the virtual scene at the current time step to obtain a target recognition result, wherein the target recognition result includes the category of the virtual object in the image and the position of the bounding box of the virtual object.

[0137] In some embodiments, the specific process of target identification refers to the description of steps 10111 to 10112 above.

[0138] In step 10132, a segmentation mask for each pixel in the bounding box is obtained, and the segmentation mask is applied to the pixel of the image to form a binary mask image of the image, wherein the segmentation mask is used to characterize whether the pixel belongs to a virtual object instance.

[0139] In some embodiments, a deep learning algorithm (such as Mask R-CNN, etc.) can be used to confirm whether the current pixel belongs to the foreground or the background. When the pixel is the foreground, the segmentation mask can be set to 1, and when the pixel is the background, the segmentation mask can be set to 0 to form a binary mask image of the image. Here, confirming whether the current pixel belongs to the foreground or the background can be understood as a binary classification task. For the classification of pixels, please refer to the description of step 10121 above.

[0140] For example, assuming there is a virtual object in the image, a segmentation mask of 1 indicates that the pixel is the pixel of the virtual object in the image (the corresponding pixel represents the foreground of the image), and a segmentation mask of 0 indicates that the pixel is the pixel corresponding to the background in the image (the corresponding pixel represents the background of the image).

[0141] In step 10133, text generation processing is performed on the binary mask image to obtain an instance segmentation result.

[0142] In some embodiments, text generation can be performed based on a binary mask image of an image using a large language model, as shown in the following formula:

[0143] y3=f(x,θ)=GPT(x,Encode(x)) (3)

[0144] Where x represents the input image, y3 represents the output instance segmentation result, and θ represents the model parameters. Encode represents a neural network encoding model (such as Mask R-CNN or RetinaNet). It generates a matrix of the same size as the image, where each element represents the encoding value (corresponding to the segmentation mask) of the corresponding pixel. GPT represents a ChatGPT-based image-to-text generation model. It generates a suitable text description based on the image and encoding results (e.g., the output of the neural network encoding model is used as the input of GPT. GPT parses and encodes the input information, and after initialization, it gradually predicts the next word and combines the generated text content to generate a coherent text sequence). The description includes the category and location of the target (virtual object). For example, for an image of a street scene containing a game character and a virtual car, the output text description might be: "In this image, a young man in a blue shirt can be seen crossing the street. A red car with a roof rack is parked on the surrounding street. The street is flanked by trees and a few white clouds float in the sky."

[0145] Continue to see Figure 4B In step 1014, the pose of the image of the virtual scene at the current time step is estimated to obtain a pose estimation result, wherein the pose estimation result includes the coordinates of multiple key points of the virtual object in the image and the connection relationship between the multiple key point coordinates.

[0146] In some embodiments, see Figure 4F , Figure 4B Step 1014 shown can be implemented by following steps 10141 to 10143, which are described in detail below.

[0147] In step 10141, multiple key point coordinates of each virtual object in the image of the virtual scene at the current time step are obtained, and the connection relationship between the multiple key point coordinates is obtained.

[0148] In some embodiments, a deep learning algorithm (such as DeepPose, DeepCut, etc.) can be used to obtain multiple key point coordinates of each virtual object in the image of the virtual scene at the current time step, and obtain the connection relationship between the multiple key point coordinates.

[0149] For example, see Figure 9B , Figure 9B This is a schematic diagram of the framework of posture estimation provided by an embodiment of the present application. First, feature extraction is performed on the input image (for example, image features are extracted through a convolutional neural network). Based on the feature extraction, key point detection is performed through models such as convolutional neural networks. Taking the posture estimation of a game character as an example, each key point represents a specific part of the human body, such as the head, shoulders, arms, knees, etc. The key points can be detected by predicting the position coordinates of each key point through a regression model. For example, the key point coordinates can be expressed as (x, y, v), where x and y represent the key point coordinates, and v indicates whether the key point is visible. After obtaining the key point position, the key points can be connected according to the connection relationship between the key points. The connection relationship can be modeled through a predefined bone structure. The posture can be reconstructed based on the key points and the connection relationship, including determining the spatial position, angle and relative posture between the joints of the key points, etc., thereby inferring the posture of the human body by comprehensively considering the key points and the connection relationship.

[0150] In step 10142, multiple key point coordinates and connection relationships are combined into posture information of each virtual object.

[0151] In step 10143, text generation processing is performed on the posture information to obtain a posture estimation result.

[0152] In some embodiments, text generation can be performed based on gesture information using a large language model, as shown in the following formula:

[0153] y4=f(x,θ)=GPT(x,Action(x)) (4)

[0154] Where x represents the input image, y4 represents the output pose estimation result, and θ represents the model parameters. Action represents a pose estimation model (such as DeepPose or DeepCut), which generates a data structure containing the coordinates and connection relationships of key points based on an image. GPT represents a ChatGPT-based image-text generation model, which generates a suitable text description containing the pose and action of a virtual object based on the image and the output of the pose estimation model.

[0155] Continue to see Figure 4B In step 1015, at least two of the target recognition result, the semantic segmentation result, the instance segmentation result, and the pose estimation result are combined into text description information.

[0156] Through step 101, it is possible to obtain text description information of the image through a large language model and a visual model (corresponding to YOLO, DeepPose, etc. mentioned above). The text information contains information such as the location, category, and posture of the virtual object, avoiding the problem in related technologies that UI controls cannot be identified due to the lack of clear attributes or identification, and improving the ability to understand and recognize virtual objects contained in the image.

[0157] Continue to see Figure 4A ,In step 102, the semantic understanding information of the virtual scene in the previous time step is obtained.

[0158] In some embodiments, the database stores action parameters and semantic understanding information for each executed time step, and the semantic understanding information of the virtual scene at the previous time step is queried from the database.

[0159] In step 103, the text description information and the semantic understanding information of the previous time step are encoded to obtain the semantic understanding information of the virtual scene at the current time step.

[0160] Continuing with the previous example, we can encode the text description information obtained based on the large language model and the semantic understanding information of the previous time step to obtain the semantic understanding information of the virtual scene at the current time step. The formula is as follows:

[0161] H t =RNN(S ChatGPT ,H t-1 ) (5)

[0162] Among them, H t Represents the semantic understanding information of the virtual scene at the current time step, S ChatGPT Represents the text description information obtained based on a large language model (such as ChatGPT, etc.), H t-1 Represents the semantic understanding information of the previous time step, and RNN represents the recurrent neural network model.

[0163] In some embodiments, see Figure 4G , Figure 4A The illustrated step 103 can be implemented by following the steps 1031 to 1032, which are described in detail below.

[0164] In step 1031, a first linear transformation is performed on the text description information to obtain a first linear result, and a second linear transformation is performed on the semantic understanding information of the previous time step to obtain a second linear result.

[0165] In some embodiments, a first linear transformation is performed on the text description information using a preset first weight parameter to obtain a first linear result, and a second linear transformation is performed on the semantic understanding information of the previous time step using a preset second weight parameter to obtain a second linear result, which can be expressed by the following formula:

[0166] H t =f(W*X t +U*H t-1 +b) (6)

[0167] Among them, f is the activation function, W is the weight parameter from the input layer to the hidden layer (corresponding to the first weight parameter), U is the weight parameter from the hidden layer to the hidden layer (corresponding to the second weight parameter), b is the bias parameter, X t is the input text description information, H t-1 is the semantic understanding information of the previous time step, W*X t The first linear transformation is performed on the text description information, U*H t-1 A second linear transformation is performed on the semantic understanding information of the previous time step.

[0168] In step 1032, the sum of the first linear result, the second linear result and the preset bias parameter is determined, and the sum is subjected to nonlinear transformation processing to obtain semantic understanding information of the current time step.

[0169] In some embodiments, referring to formula (6), the sum is processed by a nonlinear transformation using a preset activation function f (such as a tanh function) to obtain the semantic understanding information H of the current time step. t .

[0170] Continue to see Figure 4A ,In step 104, the action parameters of the virtual object in the previous time step are obtained, and the feature information of the virtual object in the current time step is obtained.

[0171] Here, taking the game scene as an example, the feature information of the current time step may be, for example, the identity identifier, remaining health, and the number of currently equipped ammunition of the game character at the current time step.

[0172] In some embodiments, obtaining the action parameters of the virtual object in the previous time step can be achieved by storing the action parameters and semantic understanding information of each executed time step in a database, and querying the action parameters of the previous time step from the database.

[0173] In other embodiments, see Figure 3DThe memory network includes a memory matrix. Each row of the memory matrix represents a memory unit. Each memory unit includes historical information (such as action parameters in previous time steps). Obtaining the action parameters of the virtual object in the previous time step can be achieved in the following ways: retrieving and reading the action parameters of the previous time step in the memory matrix, such as calculating the similarity between the query vector and each memory unit, and obtaining the reading result according to the weighted similarity. By introducing the memory network, the beneficial effect of helping the action model better understand the context information and continuously learning from historical action parameters is achieved, thereby improving the prediction accuracy of the action parameters.

[0174] Here, the similarity between the query vector and each memory unit can be calculated using cosine similarity, inner product, etc. Next, the obtained similarity value can be converted into a probability distribution (probability value is weight value) through the softmax function to ensure that the sum of the weights is 1. Each memory unit is multiplied by the corresponding weight value, and the results are added to obtain the final reading result.

[0175] In step 105, action prediction processing is performed using the semantic understanding information of the current time step, the feature information at the current time step, and the action parameters of the previous time step to obtain the action parameters of the virtual object at the current time step, wherein the action parameters of the current time step are used to drive the virtual object to perform an action at the current time step.

[0176] In some embodiments, the action parameters of the virtual object at the current time step can be expressed by the following formula:

[0177] A t ,M t =ActionModel(P objects ,H t ,M t-1 ) (7)

[0178] Among them, A t Indicates the execution action in the current time step action parameter (such as the escape and skill release of the game character in the game), M t Indicates the execution state of the action parameters in the current time step, ActionModel represents the action model described above, P objects Represents the feature information of the virtual object, H t Represents the semantic understanding information of the current time step, M t-1 Indicates the execution state of the action parameters in the previous time step, specifically the state of each virtual object. Taking games as an example, the execution state can be the state of the game character's skill release (such as skill release completed, skill cooldown, etc.) or activity state (such as resurrection, waiting for teammates to rescue, etc.).

[0179] In some embodiments, after obtaining the motion parameters of the virtual object at the current time step, the motion parameters and semantic understanding information of the current time step may be stored in a database to facilitate subsequent acquisition of historical data of the motion parameters and semantic understanding information.

[0180] In some embodiments, the action parameters of the current time step include the execution action and execution status of the current time step. After obtaining the action parameters of the virtual object at the current time step, the execution action of the current time step can also be executed on the virtual object until the execution state is reached. Taking the game as an example, for example, the execution action is to release the first skill, and the execution status is that the virtual object is releasing the first skill; the execution action is to escape to the east, and the execution status is to move to the east, and the movement speed is increased to twice the original speed; the execution action is to release the first skill, and the execution status is that the virtual object has completed the release of the first skill, etc.

[0181] In some embodiments, see Figure 4H , in execution Figure 4A Before step 105 shown, steps 106 to 110 may be performed to train the motion model, as described in detail below.

[0182] In step 106 , a training data set is obtained, wherein the training data set includes a plurality of training samples and a label of each training sample, and the plurality of training samples conform to different manipulation strategies.

[0183] In some embodiments, see Figure 4I , Figure 4H The illustrated step 106 can be implemented by following the steps 1061 to 1062, which are described in detail below.

[0184] In step 1061 , record data of multiple historical interaction processes of the virtual scene is obtained, wherein the multiple historical interaction processes are implemented based on different manipulation strategies.

[0185] In some embodiments, corresponding control strategies can be formulated according to business logic or business rules. Taking the game scene as an example, when multiple game characters compete with each other, the game characters perform control strategies such as escaping and releasing skills according to the opponent's health value (such as blood volume, etc.) and their own status (such as the skills that can be released at present, blood volume and the distance from the opponent, etc.); when the game character's health value is low to a preset value, the game character's movement speed or attack power is reduced; when an advertising pop-up window is encountered when logging into the game, the pop-up window is automatically closed, so that the game can continue to log in; the control strategy of automatically releasing skills by setting the game skill priority, such as determining the release order by factors such as skill damage and cooling time.

[0186] In step 1062, data of multiple time steps are sampled from the recorded data, multiple training samples are constructed based on the data of the multiple time steps, and the action parameters of the next time step corresponding to each training sample are used as the label of each training sample.

[0187] In some embodiments, data of multiple time steps (text description information, feature information, action parameters of the previous time step) are sampled from the recorded data, multiple training samples are constructed based on the data of multiple time steps, and the action parameters of the next time step corresponding to each training sample are used as labels for each training sample.

[0188] In some embodiments, multiple training samples can be represented as: ([image: 001, label: first skill being released, operation strategy: attack], [image: 002, label: first skill released, operation strategy: attack], [image: 003, label: first skill cooling down, operation strategy: attack]).

[0189] In some embodiments, see Figure 5A , Figure 5A This is a first schematic diagram of the construction principle of the training sample provided in the embodiment of the present application. Figure 5A Two game characters (game character A and game character B) are shown. At time step t-1, game character A releases the first skill to game character B and hits game character B. At time step t, due to encountering the first skill of game character B, the health value of game character A drops from 90% remaining to 50% remaining. In time step t+1, game character A makes an escape strategy judgment based on its own remaining blood volume, thereby moving away from game character A. Through the above example, data from multiple time steps are sampled, and multiple training samples are constructed. The action parameters of the next time step corresponding to each training sample are used as the label of each training sample. Here, the construction principle of the training sample is only used as an example. Relevant data can be sampled and labeled accordingly according to specific business rules.

[0190] In some embodiments, see Figure 5B , Figure 5B This is a second schematic diagram of the construction principle of the training sample provided in the embodiment of the present application. Figure 5B The game login interface is shown. At time step t-1, the user enters the game login interface. At time step t, an advertisement pop-up window appears. At time step t-1, the advertisement pop-up window is closed and the game login operation continues. Through this training sample sampling method, the action model can learn to deal with emergencies, and then make more intelligent decisions and output corresponding action parameters. Compared with related technologies, the ability of virtual objects to adapt to complex scenarios in the automation process is improved.

[0191] Continue to see Figure 4H ,In step 107, the training data set is divided into multiple batches according to different manipulation strategies, and the following processing is performed on each batch.

[0192] In some embodiments, the correspondence between training samples and strategies can be one-to-one or one-to-many.

[0193] In some embodiments, the training data set can be divided according to the control strategy so that the samples in each batch conform to the same operation strategy. One control strategy corresponds to at least one batch. If one control strategy corresponds to multiple batches, the multiple batches are adjacent, so that the motion model can focus on learning one control strategy each time it is trained, thereby improving the motion parameter prediction accuracy of the motion model.

[0194] In step 108, a motion prediction result for each training sample is generated using the motion model.

[0195] In some embodiments, see Figure 3D , input training samples, each training sample includes the text description information of the image of the virtual scene at the current time step (obtained through the above-mentioned large language model), the feature information P of the virtual object objects and the action parameters at the previous time step (see formula (7)). In the input layer, the text information is processed through word embedding and converted into a text vector representation. In the intermediate layer, the text vector representation is gradually read through a temporal network (such as RNN, Long Short-term Memory Networks (LSTM), Transformer, etc.) and converted into a hidden state representation. Finally, the hidden state representation is mapped to the action prediction result of the current time step through the output layer (such as a fully connected layer).

[0196] In step 109, the loss value corresponding to the preset loss function is determined through the label and action prediction result of each training sample.

[0197] In some embodiments, the loss value of a corresponding preset loss function (eg, a cross entropy loss function, a logarithmic loss function, etc.) is determined based on the label and action prediction result of each training sample.

[0198] In step 110 , model parameters of the motion model are updated based on the loss value.

[0199] In some embodiments, the gradient of the loss function with respect to the action model parameters is calculated based on the loss value through the back propagation algorithm (BP), and the model parameters of multiple network layers of the action model are updated through the gradient descent method (such as batch gradient descent (BGD), stochastic gradient descent (SGD), etc.) according to the direction of the gradient.

[0200] In some embodiments, Figure 4A Before step 101 shown, the image may be subjected to image preprocessing operations, wherein the image preprocessing operations include at least one of the following: compressing and cropping the image; performing image enhancement processing on the image; normalizing the image; encoding the image according to a preset encoding method.

[0201] In some embodiments, in order to reduce the amount of computation and memory usage, the size and quality of the image may be compressed and cropped to retain the main content and features of the image and remove redundant edges and background.

[0202] In some embodiments, the image can be encoded in different ways, such as color model (Hue-Saturation-Value, HSV), grayscale, etc. The encoding method of the image will affect the recognition effect of the large language model, so it is necessary to select a suitable encoding method according to different business types and scenarios. At the same time, in order to enable users to understand the output of the large language model, the numerical matrix can be converted into the form of an image, that is, the decoding of the image. The decoding of the image is to perform the corresponding inverse transformation according to the encoding method of the image. The encoding of the image is shown as follows:

[0203] X=f(I)=σ(W*I+b) (8)

[0204] Where I is the input image, f() is the encoding function, which converts the image I into a H×W×D tensor X, where D is the encoding dimension, W is a D×C weight matrix, b is a D-dimensional bias vector, and σ is an activation function such as tanh or ReLU.

[0205] Recover the original image I from X, that is, the decoding formula of the image is as follows:

[0206] I=g(X)=σ(W′*X+b′) (9)

[0207] Where W′ is a C×D weight matrix, b′ is a C-dimensional bias vector, and σ is the same activation function as that used in image encoding. Formula (9) is used to map the D encoding values ​​of each pixel back to C channel values, resulting in a tensor I that is close to the original image.

[0208] In some embodiments, image enhancement involves improving image quality and contrast through transformations and filtering, increasing image diversity and complexity, and thereby increasing the training data and difficulty of large language models. Image enhancement can be achieved through various methods, such as rotation, scaling, translation, cropping, flipping, noise enhancement, blurring, sharpening, and color conversion.

[0209] In some embodiments, image normalization involves unifying and standardizing the range and distribution of the image's numerical matrix through mathematical operations, thereby eliminating differences and interference between images and improving the computational efficiency and stability of large language models. Image normalization can be performed using various methods, such as maximum-minimum normalization, mean-variance normalization, and histogram equalization.

[0210] Through steps 101 to 105, text generation of a large language model, semantic understanding of the context, and long-term memory of the action model are achieved. An automation solution for virtual objects that combines temporal context semantic understanding is designed, achieving the beneficial effect of intelligent and automated control of virtual objects.

[0211] Below, we will combine the exemplary application and implementation of the server and terminal device provided in the embodiment of the present application to illustrate the processing method of virtual objects provided by the embodiment of the present application in the game scenario. In the field of gaming, the processing of virtual objects is very important. For example, in the process of game development, automated testing can be performed through the automated control of virtual objects, thereby reducing the investment cost of automated testing. Figure 6 , Figure 6 This is a schematic diagram of the game automation framework provided by an embodiment of the present application. By taking a screenshot of the current game interface, an image of the current time step is obtained (corresponding to the image of the virtual environment at the current time step), and the game state corresponding to the current time step is obtained (corresponding to the feature information of the virtual object above). The image of the current time step is understood through a language-vision large model (corresponding to step 101 above, the text description information of the image is obtained through the large language model and the visual model), and the state is encoded in combination with the context (corresponding to the semantic understanding information above). The action parameters corresponding to the current time step are retrieved from the historical memory in combination with the action agent (Agent), the action is executed through the action parameters, and the action parameters and feature information of the current time step are recorded in real time to the memory bank (corresponding to the database or memory network above), thereby achieving long-term memory and improving the generalization ability of the model.

[0212] See also Figure 7 , Figure 7 This is a schematic diagram of the interactive process of game automation provided by the embodiment of the present application, which will be combined with Figure 7 The steps shown are explained.

[0213] In step 201, the terminal device sends an image of the game interface at the current time step, where the image includes at least one game entity.

[0214] In some embodiments, game entities (corresponding to the virtual objects mentioned above) refer to interactive objects in the game world, including game characters, non-character players, props, buttons, triggers, and other elements that can control and affect the game state.

[0215] In step 202, the server obtains text description information of the image.

[0216] In some embodiments, text description information of the image is obtained through a large language model. Here, the acquisition of text information refers to the description of step 101 above.

[0217] In some embodiments, see Figure 8 , Figure 8 This is a schematic diagram of a game image provided by an embodiment of the present application. The large language model can understand the semantics of the question and the image and generate text description information for the specified game entity, such as Figure 8 As shown, the game image includes a "Start Game Button". When the game image is input to the large language model and the question "Read this image and find the percentage position of the center position of the text "Start Game" in the image as a percentage of the width and height of the image" is input, the text description information output by the large language model can be "The width of the image is W and the height is H. The coordinates of the upper left corner of the text "Start Game" are (x1, y1), and the coordinates of the lower right corner are (x2, y2). The coordinates of the center position of the text "Start Game" are (x c ,yc), where The center position of the text "Start Game" is located at the percentage position of the width and height of the picture (pw, ph), where Based on the observation of the picture, the following values ​​are estimated: W≈800, H≈600. (x1,y1)≈(300,500), (x2,y2)≈(500,550). (x c ,y c )≈400,525). (pw,ph)≈(0.5,0.875). Therefore, the estimated center position of the "Start Game" text in the image is at the percentage of the image's width and height (0.5,0.875). This information can be used to locate the corresponding position of the "Start Game" button on the game interface, thereby triggering the "Start Game" button to execute the corresponding command.

[0218] In step 203, the server obtains semantic understanding information of the image at the previous time step.

[0219] Here, for the specific implementation method of obtaining the semantic understanding information of the image at the previous time step, please refer to the description of step 102 above, and will not be repeated here.

[0220] In step 204, the server obtains the semantic understanding information of the image at the current time step through the text description information and the semantic understanding information of the previous time step.

[0221] Here, for the specific implementation method of obtaining the semantic understanding information of the image at the current time step through the text description information and the semantic understanding information of the previous time step, please refer to the description of step 103 above, which will not be repeated here.

[0222] In step 205, the server obtains the action parameters of the previous time step and obtains the feature information of the game entity.

[0223] Here, for the specific implementation method of obtaining the action parameters of the previous time step and obtaining the characteristic information of the game entity (virtual object), please refer to the description of step 104 above, which will not be repeated here. Taking the game entity as a game character as an example, the characteristic information may include the current game character's identity (such as ID, etc.), professional attributes (including attribute bonuses obtained by the game character through game skins, such as increasing the game character's movement speed attribute by wearing game skins, etc.), remaining blood volume, the number of currently equipped ammunition, etc.

[0224] In step 206, the server performs action prediction processing based on the semantic understanding information and feature information of the current time step and the action parameters of the previous time step to obtain the action parameters of the game entity at the current time step.

[0225] Here, for the specific implementation of obtaining the action parameters of the game entity (virtual object) at the current time step, please refer to the description of step 105 above, which will not be repeated here.

[0226] In step 207, the server sends action parameters.

[0227] In step 208, the terminal device performs an action.

[0228] In some embodiments, the action parameters of the current time step include the execution action and execution status of the current time step. The terminal device performs corresponding actions on the game entity according to the action parameters. For example, the execution action is to release the first skill, and the execution status is that the first skill of the virtual object has been released.

[0229] Through steps 201 to 208, it is achieved that the image of the game interface at the current time step is read to obtain the text description information of the image, and the semantic understanding information of the context at the current time step is obtained by modeling the time context. Finally, the action parameters of the game entity at the current time step are obtained through the semantic understanding information of the current time step, the feature information of the game entity (corresponding to the virtual object above) and the action parameters of the previous time step, thereby driving the game entity to perform the corresponding action at the current time step, achieving the beneficial effect of intelligently automating the action of the game entity based on the information in the image and the modeling of the time context.

[0230] The following continues to describe the exemplary structure of the virtual object processing device 133 provided in the embodiment of the present application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the virtual object processing device 133 of the memory 130 may include:

[0231] A text information generation module 1331 is used to obtain text description information of an image of a virtual scene at a current time step, wherein the image includes at least one virtual object; a context semantic understanding module 1332 is used to obtain semantic understanding information of the virtual scene at a previous time step; in some embodiments, the context semantic understanding module 1332 is further used to encode the text description information and the semantic understanding information of the previous time step to obtain semantic understanding information of the virtual scene at the current time step; an action generation module 1333 is used to obtain action parameters of the virtual object at the previous time step and obtain feature information of the virtual object at the current time step; in some embodiments, the action generation module 1333 is further used to perform action prediction processing through the semantic understanding information of the current time step, the feature information at the current time step and the action parameters of the previous time step to obtain the action parameters of the virtual object at the current time step, wherein the action parameters of the current time step are used to drive the virtual object to perform an action at the current time step.

[0232] In some embodiments, the contextual semantic understanding module 1332 is further used to perform a first linear transformation on the text description information to obtain a first linear result, and to perform a second linear transformation on the semantic understanding information of the previous time step to obtain a second linear result; determine the sum of the first linear result, the second linear result and a preset bias parameter, and perform a nonlinear transformation on the sum to obtain the semantic understanding information of the current time step.

[0233] In some embodiments, the action prediction processing is implemented by calling an action model, and the action generation module 1333 is also used to obtain a training data set, wherein the training data set includes multiple training samples and a label for each training sample, and the multiple training samples conform to different control strategies; the training data set is divided into multiple batches according to the different control strategies, and the following processing is performed on each batch: the action prediction result of each training sample is generated through the action model; the loss value of the corresponding preset loss function is determined through the label of each training sample and the action prediction result; and the model parameters of the action model are updated based on the loss value.

[0234] In some embodiments, the action generation module 1333 is also used to obtain recorded data of multiple historical interaction processes of the virtual scene, wherein the multiple historical interaction processes are implemented based on the different manipulation strategies; sample data of multiple time steps from the recorded data, construct the multiple training samples based on the data of the multiple time steps, and use the action parameters of the next time step corresponding to each training sample as the label of each training sample.

[0235] In some embodiments, the text information generation module 1331 is further used to perform target recognition on the image of the virtual scene at the current time step to obtain a target recognition result, wherein the target recognition result includes the category of the virtual object in the image and the position of the bounding box of the virtual object; perform semantic segmentation on the image of the virtual scene at the current time step to obtain a semantic segmentation result, wherein the semantic segmentation result includes at least one region in the image and the category of the region; perform instance segmentation on the image of the virtual scene at the current time step to obtain an instance segmentation result, wherein the instance segmentation result includes the category of the virtual object in the image and the mask area of ​​the virtual object; perform pose estimation on the image of the virtual scene at the current time step to obtain a pose estimation result, wherein the pose estimation result includes multiple key point coordinates of the virtual object in the image and the connection relationship between the multiple key point coordinates; and combine at least two of the target recognition result, the semantic segmentation result, the instance segmentation result and the pose estimation result into text description information.

[0236] In some embodiments, the text information generation module 1331 is also used to perform convolution processing on the image of the virtual scene at the current time step to obtain a feature representation of the image; perform classification regression processing on the feature representation to obtain the category of the virtual object and the position of the bounding box; perform text generation processing on the category and the position to obtain a target recognition result.

[0237] In some embodiments, the text information generation module 1331 is also used to obtain the category label of each pixel in the image; according to the category label of each pixel in the image, the image is segmented into regions with different category labels, wherein different category labels represent different semantics; and text generation processing is performed on the regions with different category labels to obtain semantic segmentation results.

[0238] In some embodiments, the text information generation module 1331 is also used to perform target recognition on the image of the virtual scene at the current time step to obtain a target recognition result, wherein the target recognition result includes the category of the virtual object in the image and the position of the bounding box of the virtual object; obtain a segmentation mask for each pixel point in the bounding box, and apply the segmentation mask to the pixel point of the image to form a binary mask image of the image, wherein the segmentation mask is used to characterize whether the pixel point belongs to the virtual object instance; perform text generation processing on the binary mask image to obtain the instance segmentation result.

[0239] In some embodiments, the text information generation module 1331 is also used to obtain multiple key point coordinates of each virtual object in the image of the virtual scene at the current time step, and obtain the connection relationship between the multiple key point coordinates; combine the multiple key point coordinates and the connection relationship into posture information of each virtual object; perform text generation processing on the posture information to obtain a posture estimation result.

[0240] In some embodiments, the database stores the action parameters and semantic understanding information of each executed time step, and the action generation module 1333 is also used to query the action parameters of the previous time step from the database; query the semantic understanding information of the virtual scene in the previous time step from the database; and store the action parameters of the current time step and the semantic understanding information of the current time step in the database.

[0241] In some embodiments, the text information generation module 1331 is further used to perform image preprocessing operations on the image, wherein the image preprocessing operations include at least one of the following: compressing and cropping the image; performing image enhancement processing on the image; normalizing the image; encoding the image according to a preset encoding method.

[0242] In some embodiments, the action generation module 1333 is further configured to execute the execution action of the current time step on the virtual object until the execution state is reached.

[0243] An embodiment of the present application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the virtual object processing method described in the embodiment of the present application.

[0244] The embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will execute the virtual object processing method provided by the embodiment of the present application, for example, Figure 4A A method for processing virtual objects is shown.

[0245] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0246] In some embodiments, computer-executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0247] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0248] By way of example, computer-executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed across multiple sites and interconnected by a communication network.

[0249] To sum up, through the embodiments of the present application, based on the action parameters of the virtual object in the previous time step, various information of the current time step of the virtual scene (including semantic understanding information of the image of the virtual scene in the current time step and feature information of the virtual object in the current time step) can be fully utilized to perform action prediction, so that the predicted action parameters of the current time step can match the context of the scene and the characteristics of the virtual scene itself. Therefore, when facing various complex virtual scenes, the action parameter prediction of the virtual object can be realized in units of time steps, thereby ensuring the continuity and accuracy of the action of the virtual object.

[0250] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A method for processing a virtual object, characterized in that: The method comprises: Obtaining text description information of an image of a virtual scene at a current time step, wherein the image includes at least one virtual object; Obtaining semantic understanding information of the virtual scene at a previous time step; Obtaining semantic understanding information of the virtual scene at the current time step by encoding the text description information and the semantic understanding information of the previous time step; Obtaining the action parameters of the virtual object at the previous time step, and obtaining the feature information of the virtual object at the current time step; Action prediction processing is performed through the semantic understanding information of the current time step, the feature information at the current time step and the action parameters of the previous time step to obtain the action parameters of the virtual object at the current time step, wherein the action parameters of the current time step are used to drive the virtual object to perform action at the current time step.

2. The method according to claim 1, characterized in that The encoding of the text description information and the semantic understanding information of the previous time step to obtain the semantic understanding information of the virtual scene at the current time step includes: Performing a first linear transformation on the text description information to obtain a first linear result, and performing a second linear transformation on the semantic understanding information of the previous time step to obtain a second linear result; Determine the sum of the first linear result, the second linear result, and a preset bias parameter, perform nonlinear transformation processing on the sum, and obtain semantic understanding information of the current time step.

3. The method according to claim 1, characterized in that The action prediction process is achieved by calling an action model; Before performing action prediction processing using the semantic understanding information of the current time step, the feature information of the current time step, and the action parameters of the previous time step, the method further includes: Obtaining a training data set, wherein the training data set includes a plurality of training samples and a label of each of the training samples, and the plurality of training samples conform to different manipulation strategies; Divide the training data set into multiple batches according to the different manipulation strategies, and perform the following processing on each batch: Generate an action prediction result for each of the training samples using the action model; Determining a loss value corresponding to a preset loss function based on the label of each training sample and the action prediction result; Model parameters of the motion model are updated based on the loss value.

4. The method according to claim 3, characterized in that The obtaining of the training data set includes: Acquiring recorded data of a plurality of historical interaction processes of the virtual scene, wherein the plurality of historical interaction processes are implemented based on the different manipulation strategies; Data of multiple time steps are sampled from the recorded data, the multiple training samples are constructed based on the data of the multiple time steps, and the action parameters of the next time step corresponding to each training sample are used as labels of each training sample.

5. The method according to claim 1, wherein The obtaining of text description information of the image of the virtual scene at the current time step includes: Performing target recognition on the image of the virtual scene at the current time step to obtain a target recognition result, wherein the target recognition result includes the category of the virtual object in the image and the position of the bounding box of the virtual object; Performing semantic segmentation on the image of the virtual scene at the current time step to obtain a semantic segmentation result, wherein the semantic segmentation result includes at least one region in the image and a category of the region; Performing instance segmentation on the image of the virtual scene at the current time step to obtain an instance segmentation result, wherein the instance segmentation result includes the category of the virtual object in the image and the mask area of ​​the virtual object; Performing pose estimation on the image of the virtual scene at the current time step to obtain a pose estimation result, wherein the pose estimation result includes coordinates of multiple key points of the virtual object in the image and a connection relationship between the multiple key point coordinates; At least two of the target recognition result, the semantic segmentation result, the instance segmentation result, and the pose estimation result are combined into text description information.

6. The method according to claim 5, characterized in that The target recognition is performed on the image of the virtual scene at the current time step to obtain the target recognition result, including: Performing convolution processing on the image of the virtual scene at the current time step to obtain a feature representation of the image; Performing classification and regression processing on the feature representation to obtain the category of the virtual object and the position of the bounding box; Text generation processing is performed on the category and the position to obtain a target recognition result.

7. The method according to claim 5, characterized in that The performing semantic segmentation on the image of the virtual scene at the current time step to obtain a semantic segmentation result includes: Obtaining a category label for each pixel in the image; According to the category label of each pixel point of the image, the image is segmented into regions with different category labels, wherein different category labels represent different semantics; Text generation processing is performed on the regions with different category labels to obtain semantic segmentation results.

8. The method according to claim 5, characterized in that The performing instance segmentation on the image of the virtual scene at the current time step to obtain the instance segmentation result includes: Performing target recognition on the image of the virtual scene at the current time step to obtain a target recognition result, wherein the target recognition result includes the category of the virtual object in the image and the position of the bounding box of the virtual object; Obtaining a segmentation mask for each pixel in the bounding box, and applying the segmentation mask to the pixel of the image to form a binary mask image of the image, wherein the segmentation mask is used to indicate whether the pixel belongs to the virtual object instance; Performing text generation processing on the binary mask image to obtain the instance segmentation result.

9. The method according to claim 5, characterized in that The performing pose estimation on the image of the virtual scene at the current time step to obtain a pose estimation result includes: Obtaining a plurality of key point coordinates of each of the virtual objects in the image of the virtual scene at the current time step, and obtaining a connection relationship between the plurality of key point coordinates; Combining the plurality of key point coordinates and the connection relationship into posture information of each virtual object; Perform text generation processing on the posture information to obtain a posture estimation result.

10. The method according to any one of claims 1 to 9, characterized in that The database stores the action parameters and semantic understanding information of each executed time step. The obtaining of the action parameters of the virtual object at the previous time step includes: Query the action parameters of the previous time step from the database; The obtaining of semantic understanding information of the virtual scene at a previous time step includes: Querying the database for semantic understanding information of the virtual scene at a previous time step; After obtaining the action parameters of the virtual object at the current time step, the method further includes: The action parameters of the current time step and the semantic understanding information of the current time step are stored in the database.

11. The method according to any one of claims 1 to 9, characterized in that Before obtaining text description information of the image of the virtual scene at the current time step, the method further includes: Performing an image preprocessing operation on the image, wherein the image preprocessing operation includes at least one of the following: compressing and cropping the image; performing image enhancement processing on the image; performing normalization processing on the image; The image is encoded in a preset encoding manner.

12. The method according to any one of claims 1 to 9, characterized in that The action parameters of the current time step include the execution action and execution status of the current time step; After obtaining the action parameters of the virtual object at the current time step, the method further includes: The execution action of the current time step is executed on the virtual object until the execution state is reached.

13. A virtual object processing device, characterized in that: The device comprises: a text information generation module, configured to obtain text description information of an image of a virtual scene at a current time step, wherein the image includes at least one virtual object; A contextual semantic understanding module, used to obtain semantic understanding information of the virtual scene at the previous time step; The context semantic understanding module is further configured to encode the text description information and the semantic understanding information of the previous time step to obtain the semantic understanding information of the virtual scene at the current time step; An action generation module, configured to obtain the action parameters of the virtual object at the previous time step and obtain the feature information of the virtual object at the current time step; The action generation module is also used to perform action prediction processing based on the semantic understanding information of the current time step, the feature information at the current time step and the action parameters of the previous time step to obtain the action parameters of the virtual object at the current time step, wherein the action parameters of the current time step are used to drive the virtual object to perform action at the current time step.

14. An electronic device, characterized in that: The electronic device comprises: a memory for storing computer-executable instructions; The processor is configured to implement the virtual object processing method according to any one of claims 1 to 12 when executing the computer executable instructions stored in the memory.

15. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the method for processing a virtual object according to any one of claims 1 to 12 is implemented.

16. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the method for processing a virtual object according to any one of claims 1 to 12 is implemented.

Images