Data processing methods, apparatus, electronic devices and media

Abstract

This disclosure provides data processing methods, apparatus, electronic devices, and media, relating to the field of computer technology, and particularly to artificial intelligence technology. The implementation scheme is as follows: acquiring observation features and relative position features of multiple objects at a first time moment; determining attention coefficients for the observation features based on the relative position features; determining prediction features for the multiple objects based on the attention coefficients and the observation features; and determining the actions of the multiple objects at a second time moment based on the prediction features.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computers, in particular to artificial intelligence technology, and specifically to a data processing method and device, electronic equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] Artificial intelligence is a discipline that studies enabling computers to simulate some thinking processes and intelligent behaviors of humans (such as learning, reasoning, thinking, planning, etc.), which has both hardware and software technologies. Artificial intelligence hardware technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, etc.; artificial intelligence software technology mainly includes computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, knowledge graph technology, etc.

[0003] Utilizing multi-agent systems involves how to coordinate communication, cooperation, scheduling, management, etc. among multiple agents. Research on multi-agent systems can solve large and complex real-world problems.

[0004] The methods described in this section can not necessarily be the methods previously conceived or used. Unless otherwise indicated, nothing in this section should be assumed to be prior art merely because it is included in this section. Similarly, any problems mentioned in this section should not be assumed to have been recognized in any prior art unless otherwise indicated. SUMMARY

[0005] The present disclosure provides a data processing method and device, electronic equipment, computer readable storage medium and computer program product.

[0006] According to an aspect of the present disclosure, a data processing method is provided, comprising: obtaining observation features and relative position features of a plurality of objects at a first time; determining attention coefficients for the observation features based on the relative position features; determining prediction features for the plurality of objects based on the attention coefficients and the observation features; and determining actions of the plurality of objects at a second time based on the prediction features.

[0007] According to another aspect of the present disclosure, a data processing device is provided, comprising: an obtaining unit configured to obtain observation features and relative position features of a plurality of objects at a first time; an attention coefficient determining unit configured to determine attention coefficients for the observation features based on the relative position features; a prediction unit configured to determine prediction features for the plurality of objects based on the attention coefficients and the observation features; and a decision unit configured to determine actions of the plurality of objects at a second time based on the prediction features.

[0008] According to another aspect of the disclosure, there is provided an electronic device comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as previously described.

[0009] According to another aspect of the disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method as previously described.

[0010] According to another aspect of the disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method as previously described.

[0011] According to one or more embodiments of the disclosure, the relative distance between the individual agents can be considered more in the action decision of the multi-agent system, and the influence of the relative distance on the action decision result can be considered.

[0012] It should be understood that the matters described herein are intended to be illustrative only and are not intended to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0013] The accompanying drawings are included to provide a further understanding of embodiments and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments and together with the description serve to explain exemplary embodiments of the present disclosure. The illustrated embodiments are merely examples and are not intended to restrict the scope of the claims. In all the drawings, like reference numerals refer to like elements throughout the various figures.

[0014] Figure 1 shows a schematic diagram of an exemplary system in which the various methods described herein can be implemented according to embodiments of the disclosure;

[0015] Figure 2 shows an exemplary flowchart of a data processing method according to embodiments of the disclosure;

[0016] Figure 3 shows an exemplary process for determining an attention coefficient according to embodiments of the disclosure;

[0017] Figure 4 shows an exemplary process of a data processing method according to embodiments of the disclosure;

[0018] Figure 5 shows an exemplary block diagram of a data processing apparatus according to embodiments of the disclosure;

[0019] Figure 6 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0020] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of embodiments of the present disclosure are set forth to help understand the present disclosure. It should be understood that the embodiments described herein are merely exemplary and that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and structures are omitted.

[0021] In the present disclosure, the terms "first", "second", and the like are used to describe various elements only for the purpose of distinguishing one element from another, and the terms are not intended to limit the positions, sequence, or importance of the elements. In some examples, a first element and a second element can refer to the same instance of the element, and in some cases, they can refer to different instances of the element based on the context of the description.

[0022] The terms used in the description of various described examples in the present disclosure are only for the purpose of describing particular examples and are not intended to be limiting. Unless specifically defined otherwise, an element that is not specifically defined can be either one or more elements. Also, the term "and / or" used in the present disclosure encompasses any one of the listed items and all possible combinations of the listed items.

[0023] Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0024] Figure 1 A schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein can be implemented according to embodiments of the present disclosure is shown. Referring to Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 that couple the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 can be configured to execute one or more application programs.

[0025] In embodiments of the present disclosure, the server 120 can run one or more services or software applications that enable the execution of data processing methods for multi-agent systems according to the present disclosure.

[0026] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtual and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as to users of client devices 101, 102, 103, 104, 105, and / or 106 under a Software as a Service (SaaS) model.

[0027] exist Figure 1 In the configuration shown, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or combinations thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 can sequentially interact with server 120 using one or more client applications to utilize the services provided by these components. It should be understood that various different system configurations are possible and may differ from system 100. Therefore, Figure 1 This is an example of a system used to implement the various methods described herein, and is not intended to be limiting.

[0028] Users can use client devices 101, 102, 103, 104, 105 and / or 106 to obtain input data for the data processing method of embodiments of this disclosure. The client devices can provide an interface that allows users of the client devices to interact with them. The client devices can also output information to the user via this interface. Although Figure 1 Only six client devices are described, but those skilled in the art will understand that this disclosure can support any number of client devices.

[0029] Client devices 101, 102, 103, 104, 105, and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors, or other sensing devices. These computer devices can run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as Google Chrome OS); or include various mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablets, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, internet-enabled gaming devices, etc. Client devices are capable of executing various applications, such as various internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and can use various communication protocols.

[0030] Network 110 can be any type of network well known to those skilled in the art, and can use any of a variety of available protocols (including but not limited to TCP / IP, SNA, IPX, etc.) to support data communication. By way of example only, one or more networks 110 can be a local area network (LAN), an Ethernet-based network, a token ring network, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0031] Server 120 may include one or more general-purpose computers, special-purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0032] The computing unit in server 120 can run one or more operating systems, including any of the aforementioned operating systems and any commercially available server operating system. Server 120 can also run any of a variety of additional server applications and / or middleware applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0033] In some implementations, server 120 may include one or more applications to analyze and merge data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105 and / or 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105 and / or 106.

[0034] In some implementations, server 120 can be a server for a distributed system or a server integrated with blockchain. Server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0035] System 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. Databases 130 may reside in various locations. For example, a database used by server 120 may be local to server 120, or it may be located away from server 120 and may communicate with server 120 via a network-based or dedicated connection. Databases 130 may be of different types. In some embodiments, the database used by server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data from and from the databases in response to commands.

[0036] In some embodiments, one or more of the databases 130 may also be used by an application to store application data. The databases used by the application may be of different types, such as key-value stores, object stores, or regular stores supported by a file system.

[0037] Figure 1The system 100 can be configured and operated in various ways to enable the application of the various methods and apparatus described in this disclosure.

[0038] Multi-agent reinforcement learning can aid in decision-making for various complex systems, such as simulated adversarial or game-like scenarios involving multiple objects. It can be used to make decisions about agent actions within systems such as various sports, games, and combat systems (e.g., drones).

[0039] However, current multi-agent reinforcement learning methods, while using the coordinates of each agent within the system as input to generate the final action decision, do not emphasize the impact of the relative positions of the agents on the action decision outcome during learning. In reality, however, the relative positions of agents can be a significant determinant of the action decision. For example, in a football match, the relative positions of players can directly influence the passing choice of the player with the ball. Similarly, in games, limited by the skill range defined in the game, the relative positions of game characters can also affect their next action decision.

[0040] Figure 2 An exemplary flowchart of a data processing method according to embodiments of the present disclosure is shown. It can be utilized... Figure 1 The client or server implementation shown in the figure Figure 2 Method 200 is shown in the figure.

[0041] like Figure 2 As shown, in step S202, the observed features and relative position features of multiple objects at the first time point are acquired. In step S204, attention coefficients for the observed features are determined based on the relative position features. In step S206, predicted features for the multiple objects are determined based on the attention coefficients and the observed features. In step S208, the actions of the multiple objects at the second time point are determined based on the predicted features.

[0042] The data processing method provided by the embodiments of this disclosure can be used to predict the state of each object by utilizing the relative position features of multiple objects and the attention coefficients based on the aforementioned relative position features, thereby further obtaining action decision results for each object. By incorporating the relative position information between multiple agents into the reinforcement learning model through the attention mechanism, the agents in the system can more fully understand the current relative position information between agents, thereby making more optimized action decisions in subsequent moments.

[0043] The data processing method of the embodiments of this disclosure will be explained in more detail below.

[0044] In step S202, the observation features and relative position features of multiple objects at the first moment can be obtained.

[0045] In some embodiments, relative position features may include the distance between any two objects among multiple objects. These relative position features can be obtained by calculating the distance between any two objects (any two agents) in a multi-agent system. In some examples, the relative position features may be a discretized result of the distance between any two objects among multiple objects. This can be achieved by discretizing the actual distance calculated between any two objects in the multi-agent system and inputting the discretized result as the relative position features into the model. Discretizing the actual distance information helps the model better understand the relative distances between objects and simplifies model training and computation.

[0046] Observational features can indicate the individual state of each object within a system environment, as well as the environmental state. In some embodiments, observational features may include at least one of the following for each object among multiple objects: identity, coordinates, orientation, and state. Taking a football match as an example, observational features may include the identity identifiers of each player (e.g., forward, goalkeeper, etc.), position coordinates, fatigue state, orientation, etc., and may also include the coordinates, height, orientation, angular velocity, etc., of the football. Various appropriate data processing methods can be applied to the information involved in the observational features to facilitate model processing.

[0047] In step S204, attention coefficients for observing features can be determined based on relative position features.

[0048] In some embodiments, step S204 may include: processing the relative position features using an embedding layer to obtain a relative position vector; processing the observation features using an embedding layer to obtain an observation vector; and processing the relative position vector and the observation vector based on a multi-head attention mechanism to obtain attention coefficients.

[0049] Multi-head attention mechanisms can be used to explicitly encourage the model to pay more attention to and understand the relative positions between agents when calculating the attention coefficients used to observe features.

[0050] In some examples, the attention coefficients described above can be determined using the following steps: determining a first matrix obtained by multiplying the relative position vector by the observation vector; determining a second matrix obtained by multiplying the observation vector by the relative position vector; determining a third matrix obtained by multiplying the observation vector by the observation vector; and determining the attention coefficients based on the normalized result of the sum of the first, second, and third matrices.

[0051] Figure 3 An exemplary process for determining an attention coefficient according to embodiments of the present disclosure is shown.

[0052] like Figure 3As shown, the attention coefficient matrix 305 can be determined based on the relative position vector P and the observation vector O. At position 301, the first matrix is ​​obtained by multiplying the relative position vector P by the observation vector O. At position 302, the second matrix is ​​obtained by multiplying the observation vector O by the relative position vector P. Different multiplication results can be obtained by adjusting the positions of the vectors in the matrix multiplication. At position 303, the third matrix is ​​obtained by multiplying the observation vector O by the observation vector O. At position 304, the first, second, and third matrices obtained at positions 301, 302, and 303 are summed, and the attention coefficient matrix 305 is obtained by normalizing the summation result at position 304. For example, the attention coefficient matrix 305 can be generated based on softmax(P*E+E*P+E*E).

[0053] The relative position vector P and observation vector O are generated by processing the relative position features and observation features separately using an embedding layer. The dimensions of the relative position vector P and observation vector O can be determined by adjusting the parameters of the embedding layer, thus enabling [the desired results]. Figure 3 The attention coefficient generation process is illustrated in the diagram. In some examples, before performing matrix multiplication on the relative position vector P and the observation vector O, spatial transformations can be applied to the relative position vector P and the observation vector O to enable the model to learn the features of the relative position vector P and the observation vector O in different spaces. For example, before step 301, a first transformation can be performed on the relative position vector P to obtain the first transformed relative position vector P', and a second transformation can be performed on the observation vector O to obtain the second transformed observation vector O'. Similarly, before step 302, the relative position vector P can be transformed to obtain the transformed relative position vector P'', and the observation vector O can be transformed to obtain the transformed observation vector O''', where P''' may be different from P', and O'''' may be different from O''. Before step 303, the observation vector O can be transformed to obtain the transformed observation vectors O''' and O'''', where O'''' and O' ...

[0054] Return to reference Figure 2 In step S206, prediction features for multiple objects can be determined based on attention coefficients and observation features.

[0055] In some embodiments, the observation attention features can be determined based on the product of the attention coefficients obtained in step 204 and the observation features. Then, the observation attention features can be processed using a perceptron to obtain the predicted features. For example, a multilayer perceptron (MLP) can be used as the perceptron to process the observation attention features.

[0056] By multiplying the attention coefficient by the observed features, different weight coefficients can be assigned to each parameter in the observed features, thus obtaining predictive features for predicting the actions of multiple objects. Specifically, by adjusting the parameters in the observed features using the attention coefficient, different parameters can have different weights in the model's prediction process, thereby optimizing the model's prediction results.

[0057] In step S208, the actions of multiple objects at the second time point are determined based on the predicted features.

[0058] Various suitable models can be used to process the predicted features to determine the actions of multiple objects at the second time step. The predicted features reflect the individual information of each object at the first time step and the interaction information between the object and the environment. Based on the predicted features, the action decision results of multiple objects at the second time step (i.e., the subsequent time step) can be obtained.

[0059] In some embodiments, step S208 may include: determining a sequence of previous actions of multiple objects; processing the sequence of previous actions using an embedding layer to obtain a sequence of previous actions vectors; processing the sequence of previous actions vectors based on a self-attention mechanism to obtain action attention features; processing the predicted features and action attention features based on an attention mechanism to obtain predicted action features; and processing the predicted action features using a perceptron to obtain the actions of multiple objects.

[0060] When predicting the actions of multiple objects at a second time step, a sequence of previous actions from multiple time steps prior to the second time step can be considered simultaneously. This sequence of previous actions can be used as one of the inputs to the prediction unit. Processing the previous action sequence using an embedding layer yields a previous action vector. In some examples, a perceptron (such as an MLP) can be used to embed the previous action sequence to obtain the previous action vector. Action attention features obtained through a self-attention mechanism allow the model to better understand the previous action sequence, thus achieving better prediction results. Furthermore, the prediction features and action attention features obtained in step S206 can be processed using an attention mechanism, enabling the model to obtain the optimal action decision at the current time step, while understanding both the prediction features and the previous action sequence.

[0061] Figure 4 An exemplary process of a data processing method according to an embodiment of the present disclosure is shown.

[0062] like Figure 4As shown, observation features 402 and relative position features 401 of multiple objects at the first time step are determined. Embedding layers 403 and 404 can process the relative position features 401 and observation features 402 respectively to obtain relative position vectors and observation vectors. The attention coefficient determination unit 405 can be used to process the relative position vectors and observation vectors to obtain attention coefficients for observation features 402. This can be achieved using... Figure 3 The structure shown implements the attention coefficient determination unit 405. At 406, the attention coefficients obtained by the attention coefficient determination unit 405 are multiplied by the observed features (or observation vectors) to obtain the observation attention features. After passing through the residual structure 407, the observation attention features can be processed by the perceptron 408 (such as MLP) to obtain the predicted features 409.

[0063] Action decisions can be generated based on the previous action sequence 410 and predicted features 409. For example... Figure 4 As shown, the previous action sequence 410 is processed by the embedding layer 411 to obtain the previous action vector. Then, the attention calculation unit 412 processes the previous action vector. The attention calculation unit can include three inputs: a query vector (Q), a key vector (K), and a value vector (V). The attention calculation unit can calculate using the formula softmax(Q*K)*V. For the attention calculation unit 412, the three inputs Q, K, and V are all previous action vectors. In some examples, the three inputs of the attention calculation unit 412 can be mappings of the previous action vectors in different spaces. After the attention calculation unit 412 outputs the action attention features, they are then processed through the residual structure 413, and another attention calculation unit 414 can be used to calculate the action attention features and the predicted features. The first input of the attention calculation unit 414 can be the predicted feature (or its mapping in other spaces), and the second and third inputs can be the action attention features (or their mappings in other spaces). The attention calculation unit 414 can output the predicted action features. After passing through residual structure 415 and perceptrons 416 and 417, the predicted action 418 for the second time step can be obtained. In some examples, perceptrons 416 and 417 can be replaced with more or fewer perceptrons. After obtaining the predicted action 418 for the second time step, the previous action sequence can be updated based on the predicted action 418 for the second time step, and actions for subsequent time steps can be generated.

[0064] Figure 5 An exemplary block diagram of a data processing apparatus according to an embodiment of the present disclosure is shown.

[0065] like Figure 5As shown, the data processing device 500 may include an acquisition unit 510, an attention coefficient determination unit 520, a prediction unit 530, and a decision unit 540.

[0066] The acquisition unit 510 can be configured to acquire the observed features and relative position features of multiple objects at a first time. The attention coefficient determination unit 520 can be configured to determine the attention coefficients used for the observed features based on the relative position features. The prediction unit 530 can be configured to determine the predicted features for multiple objects based on the attention coefficients and the observed features. The decision unit 540 can be configured to determine the actions of the multiple objects at a second time based on the predicted features.

[0067] In some embodiments, the relative position feature may include the distance between any two objects among a plurality of objects. In some examples, the relative position feature may be a discretized result of the distance between any two objects among a plurality of objects.

[0068] In some embodiments, the observation features may include at least one of the following: the identity, coordinates, orientation, and state of each of the multiple objects.

[0069] In some embodiments, the attention coefficient determination unit can be configured to: process the relative position features using an embedding layer to obtain a relative position vector; process the observation features using an embedding layer to obtain an observation vector; and process the relative position vector and the observation vector based on a multi-head attention mechanism to obtain attention coefficients.

[0070] In some embodiments, processing the relative position vector and the observation vector based on the multi-head attention mechanism to obtain attention coefficients may include: determining a first matrix obtained by multiplying the relative position vector by the observation vector; determining a second matrix obtained by multiplying the observation vector by the relative position vector; determining a third matrix obtained by multiplying the observation vector by the observation vector; and determining the attention coefficients based on the normalization result of the sum of the first matrix, the second matrix, and the third matrix.

[0071] In some embodiments, the prediction unit may be configured to: determine the observation attention features based on the product of the attention coefficient and the observation features; and process the observation attention features using a perceptron to obtain the prediction features.

[0072] In some embodiments, determining the actions of multiple objects at a second time step based on predicted features includes: determining a sequence of previous actions of the multiple objects; processing the sequence of previous actions using an embedding layer to obtain a sequence of previous actions vectors; processing the sequence of previous actions using a self-attention mechanism to obtain action attention features; processing the predicted features and action attention features using an attention mechanism to obtain predicted action features; and processing the predicted action features using a perceptron to obtain the actions of the multiple objects.

[0073] In some embodiments, embedding a previous action sequence to obtain a previous action vector may include: embedding the previous action sequence using a perceptron to obtain a previous action vector.

[0074] Can be used Figure 5 Units 510-540 shown in the figure are executed. Figure 2 Steps S202 to S208 shown in the figure will not be described again here.

[0075] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0076] According to embodiments of this disclosure, an electronic device, a readable storage medium, and a computer program product are also provided.

[0077] refer to Figure 6 The present invention describes a structural block diagram of an electronic device 600 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0078] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0079] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, output unit 607, storage unit 608, and communication unit 609. Input unit 606 can be any type of device capable of inputting information to electronic device 600. Input unit 606 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device, and can include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 607 can be any type of device capable of presenting information, and can include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 608 can include, but is not limited to, disk and optical disk. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and can include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0080] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as method 200. For example, in some embodiments, method 200 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of method 200 described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform method 200 by any other suitable means (e.g., by means of firmware).

[0081] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0082] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0083] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0084] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0085] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0086] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0087] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0088] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A data processing method, comprising: obtaining observation features and relative position features of a plurality of objects at a first time; determining attention coefficients for the observation features based on the relative position features; determining prediction features for the plurality of objects based on the attention coefficients and the observation features; determining actions of the plurality of objects at a second time based on the prediction features, wherein determining actions of the plurality of objects at a second time based on the prediction features comprises: determining a sequence of previous actions of the plurality of objects; processing the sequence of previous actions using an embedding layer to obtain a previous action vector; processing the previous action vector based on a self-attention mechanism to obtain an action attention feature; processing the prediction features and the action attention feature based on an attention mechanism to obtain a predicted action feature; processing the predicted action feature using a perception to obtain the actions of the plurality of objects.

2. The data processing method of claim 1, wherein, The relative position features comprise distances between any two objects of the plurality of objects.

3. The data processing method of claim 2, wherein the relative position features are discretized results of distances between any two objects of the plurality of objects.

4. The data processing method of claim 1, wherein, The observation features comprise at least one of identities, coordinates, directions, and states of each object of the plurality of objects.

5. The data processing method of claim 1, wherein, Determining attention coefficients for the observation features based on the relative position features comprises: processing the relative position features using an embedding layer to obtain a relative position vector; processing the observation features using an embedding layer to obtain an observation vector; processing the relative position vector and the observation vector based on a multi-head attention mechanism to obtain the attention coefficients.

6. The data processing method of claim 5, wherein, Processing the relative position vector and the observation vector based on a multi-head attention mechanism to obtain the attention coefficients comprises: determining a first matrix obtained by multiplying the relative position vector by the observation vector; determining a second matrix obtained by multiplying the observation vector by the relative position vector; determining a third matrix obtained by multiplying the observation vector by the observation vector; determining the attention coefficients based on a normalized result of a sum of the first matrix, the second matrix, and the third matrix.

7. The data processing method of any one of claims 1-6, wherein, Determining prediction features for the plurality of objects based on the attention coefficients and the observation features comprises: determining observation attention features of the observation features based on products of the attention coefficients and the observation features; processing the observation attention features using a perception to obtain the prediction features.

8. The data processing method of claim 1, wherein, Embedding the sequence of previous actions to obtain a previous action vector comprises: embedding the sequence of previous actions using a perception to obtain the previous action vector.

9. A data processing apparatus, comprising: an obtaining unit configured to obtain observation features and relative position features of a plurality of objects at a first time; an attention coefficient determining unit configured to determine attention coefficients for the observation features based on the relative position features; a prediction unit configured to determine prediction features for the plurality of objects based on the attention coefficients and the observation features; a decision unit configured to determine, based on the predicted feature, an action of the plurality of objects at a second time, wherein determining, based on the predicted feature, the action of the plurality of objects at the second time comprises: determining a previous action sequence of the plurality of objects; processing the previous action sequence using an embedding layer to obtain a previous action vector; processing the previous action vector based on a self-attention mechanism to obtain an action attention feature; processing the predicted feature and the action attention feature based on an attention mechanism to obtain a predicted action feature; processing the predicted action feature using a perception to obtain the action of the plurality of objects.

10. The data processing apparatus of claim 9, wherein, The relative position feature comprises a distance between any two objects of the plurality of objects. 11.The data processing apparatus of claim 10, wherein the relative position feature is a discretization result of a distance between any two objects of the plurality of objects.

12. The data processing apparatus of claim 9, wherein, The observation feature comprises at least one of an identity, a coordinate, a direction, a state of each object of the plurality of objects.

13. The data processing apparatus of claim 9, wherein, The attention coefficient determination unit is configured to: process the relative position feature using an embedding layer to obtain a relative position vector; process the observation feature using an embedding layer to obtain an observation vector; process the relative position vector and the observation vector based on a multi-head attention mechanism to obtain the attention coefficient.

14. The data processing apparatus of claim 13, wherein, Processing the relative position vector and the observation vector based on a multi-head attention mechanism to obtain the attention coefficient comprises: determining a first matrix obtained by multiplying the relative position vector by the observation vector; determining a second matrix obtained by multiplying the observation vector by the relative position vector; determining a third matrix obtained by multiplying the observation vector by the observation vector; determining the attention coefficient based on a normalization result of a sum of the first matrix, the second matrix, and the third matrix.

15. The data processing apparatus of any one of claims 9-14, wherein, The prediction unit is configured to: determine an observation attention feature of the observation feature based on a product of the attention coefficient and the observation feature; process the observation attention feature using a perception to obtain the predicted feature.

16. The data processing apparatus of claim 9, wherein, Embedding the previous action sequence to obtain the previous action vector comprises: embedding the previous action sequence using a perception to obtain the previous action vector. 17.An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

18. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to cause the computer to perform the method of any one of claims 1-8.

19. A computer program product comprising a computer program, wherein, The computer program, when executed by a processor, implements the method of any one of claims 1-8.

Images