Training method of behavior recognition model, behavior recognition method and related device

Abstract

The embodiment of the invention provides a behavior recognition model training method, a behavior recognition method and a related device, and relates to the field of artificial intelligence. The method comprises the steps that first information of each sample object in each sample object pair is acquired, the first information comprises object identification features of the sample objects, and the object identification features comprise at least one of behavior implementation object features and behavior bearing object features; for each sample object pair, second information of the sample object pair is determined based on the first information of each sample object in the sample object pair, and the second information comprises the similarity between the first sample object and the second sample object in the sample object pair; and training a preset model based on the second information of each sample object pair. The preset model may be a large model. According to the embodiment of the invention, the technical effect of improving the interpretable degree of model training is achieved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology. Specifically, the present application relates to a training method for a behavior recognition model, a behavior recognition method and related devices. Background Art

[0002] With the rapid development of artificial intelligence, artificial intelligence is being applied in more and more fields, for example, using artificial intelligence for behavior recognition.

[0003] In the related art, the model is mainly trained based on the implementation object characteristics of the behavior implementation object and the bearing object characteristics of the behavior bearing object, and then the trained model is used to perform behavior recognition of the object.

[0004] However, using the object features and training the model with the object features is a black box and lacks interpretability. Summary of the invention

[0005] The embodiments of the present application provide a training method for a behavior recognition model, a behavior recognition method and related devices, which are used to solve the technical problem of poor interpretability of model training and achieve the technical effect of improving the interpretability of model training.

[0006] On the one hand, an embodiment of the present application provides a method for training a behavior recognition model, comprising:

[0007] Acquire first information of each sample object in each sample object pair, the first information including an object identification feature of the sample object, the object identification feature including at least one of a behavior implementation object feature or a behavior acceptance object feature;

[0008] For each sample object pair, based on the first information of each sample object in the sample object pair, second information of the sample object pair is determined, the second information including the similarity between the first sample object and the second sample object in the sample object pair, the similarity representing at least one of the probability that the first sample object performs a behavior on the second sample object, the probability that the first sample object and the second sample object are both behavior-performing objects, the probability that the first sample object and the second sample object are both behavior-bearing objects, or the probability that the second sample object performs a behavior on the first sample object;

[0009] Based on the second information of each sample object pair, a preset model is trained until the training end condition is met to obtain a behavior recognition model. The behavior recognition model is used to determine the target behavior recognition result of the object pair to be identified. The target behavior recognition result represents whether there is a behavior implementation relationship between the first object to be identified and the second object to be identified in the object pair to be identified.

[0010] On the other hand, the embodiment of the present application also provides a behavior recognition method, including:

[0011] Acquire third information of each object to be identified in the pair of objects to be identified, the third information including an object identification feature of the object to be identified, the object identification feature including at least one of a behavior-performing object feature or a behavior-bearing object feature;

[0012] Based on the third information of each object to be identified in the pair of objects to be identified, fourth information of the pair of objects to be identified is determined, the fourth information including the similarity between the first object to be identified and the second object to be identified in the pair of objects to be identified, the similarity representing at least one of the probability that the first object to be identified implements an action on the second object to be identified, the probability that the first object to be identified and the second object to be identified are both objects implementing the action, the probability that the first object to be identified and the second object to be identified are both objects receiving the action, or the probability that the second object to be identified implements an action on the first object to be identified;

[0013] Inputting the fourth information of the pair of objects to be identified into the behavior recognition model to obtain a target behavior recognition result obtained by the behavior recognition model based on the fourth information of the pair of objects to be identified, wherein the target behavior recognition result indicates whether there is a behavior implementation relationship between the first object to be identified and the second object to be identified in the pair of objects to be identified;

[0014] Among them, the behavior recognition model is trained based on the method of any embodiment of the present application.

[0015] On the other hand, the embodiment of the present application also provides a training device for a behavior recognition model, including:

[0016] A first information acquisition module, used to acquire first information of each sample object in each sample object pair, wherein the first information includes an object identification feature of the sample object, and the object identification feature includes at least one of a behavior implementation object feature or a behavior acceptance object feature;

[0017] A first information determination module is used to determine, for each sample object pair, second information of the sample object pair based on the first information of each sample object in the sample object pair, the second information including the similarity between the first sample object and the second sample object in the sample object pair, the similarity representing at least one of the probability that the first sample object implements a behavior toward the second sample object, the probability that the first sample object and the second sample object are both behavior implementation objects, the probability that the first sample object and the second sample object are both behavior receiving objects, or the probability that the second sample object implements a behavior toward the first sample object;

[0018] The training module is used to train a preset model based on the second information of each sample object pair until the training end condition is met to obtain a behavior recognition model. The behavior recognition model is used to determine the identity recognition result of the object to be recognized. The identity recognition result includes the object that implements the behavior or the object that receives the behavior.

[0019] Optionally, the device is further used to obtain an object label of each sample object in each sample object pair, the object label representing that the sample object is a behavior implementation object or a behavior receiving object;

[0020] When training the preset model based on the second information of each sample object pair, the training module can be used to:

[0021] Using the second information of the sample object pair as an input of a preset model, so that the preset model obtains a first identity prediction result of a first sample object and a second identity prediction result of a second sample object in the sample object pair based on the second information of the sample object pair;

[0022] Determining a first prediction loss between a first identity prediction result and an object label of a first sample object, and determining a second prediction loss between a second identity prediction result and an object label of a second sample object;

[0023] If it is determined based on the first prediction loss and the second prediction loss that the training end condition is not met, updating the model parameters of the preset model and continuing to train the preset model;

[0024] If it is determined based on the first prediction loss and the second prediction loss that the training end condition is met, the training is ended, and the preset model after the training is completed is used as the behavior recognition model.

[0025] Optionally, if the object label of the sample object indicates that the sample object is a behavior-performing object, the first information of the sample object further includes the abnormal behavior characteristics of the sample object; if the object label of the sample object indicates that the sample object is a behavior-bearing object, the first information of the sample object further includes the object characteristics of the sample object;

[0026] When the training module uses the second information of the sample object pair as the input of the preset model so that the preset model obtains the first identity prediction result of the first sample object and the second identity prediction result of the second sample object in the sample object pair based on the second information of the sample object pair, it can be used to:

[0027] The second information of the sample object pair and the first information of each sample object in the sample object pair are used as inputs of a preset model, so that the preset model obtains a first identity prediction result and a second identity prediction result based on the second information of the sample object pair and the first information of each sample object in the sample object pair.

[0028] Optionally, when the first information determining module determines the second information of the sample object pair based on the first information of each sample object in the sample object pair, it can be used for at least one of the following:

[0029] Determine a first similarity between a behavior-performing object feature of the first sample object and a behavior-bearing object feature of the second sample object, the first similarity representing a probability that the first sample object performs a behavior on the second sample object;

[0030] Determine a second similarity between the behavior implementation object feature of the first sample object and the behavior implementation object feature of the second sample object, the second similarity representing the probability that the first sample object and the second sample object are both the behavior implementation objects;

[0031] Determine a third similarity between the behavior bearing object feature of the first sample object and the behavior bearing object feature of the second sample object, the third similarity representing the probability that the first sample object and the second sample object are both behavior bearing objects;

[0032] A fourth similarity between the behavior-bearing object feature of the first sample object and the behavior-implementing object feature of the second sample object is determined, the fourth similarity representing the probability that the second sample object implements the behavior toward the first sample object.

[0033] Optionally, when acquiring the first information of each sample object in each sample object pair, the first information acquisition module may be used to:

[0034] Obtaining sample object data of each sample object among a plurality of sample objects;

[0035] A feature graph is constructed based on the sample object data of each sample object, wherein the feature graph includes nodes corresponding to each sample object and connection relationships between the nodes, wherein the nodes in the feature graph store relevant features of the corresponding sample objects, and the connected nodes have association relationships, and the association relationships include friend relationships and behavior implementation relationships. The nodes in the feature graph that have behavior implementation relationships are pointed to behavior bearing object nodes by behavior implementation object nodes, and the behavior implementation object nodes also store object labels that characterize the sample object as a behavior implementation object, and the behavior bearing object nodes also store object labels that characterize the sample object as a behavior bearing object;

[0036] At least one pair of sample object pairs and first information of each sample object in each sample object pair are determined based on the feature graph, wherein the sample objects corresponding to the nodes connected in the feature graph are taken as a pair of sample object pairs.

[0037] Optionally, the sample object data includes at least one of first sample object data or second sample object data, the first sample object data includes sample object features, and the second sample object data includes abnormal behavior features;

[0038] When constructing a feature map based on the sample object data of each sample object, the first information acquisition module may be used for at least one of the following:

[0039] Building a behavior bearing object feature graph based on the first sample object data of each sample object, wherein the nodes in the behavior bearing object feature graph store sample object features of the corresponding sample object;

[0040] A behavior implementation object feature graph is constructed based on the second sample object data of each sample object, and the nodes in the behavior implementation object feature graph store the behavior abnormal features of the corresponding sample objects.

[0041] Optionally, the characteristic graph includes at least one of a behavior-bearing object characteristic graph or a behavior-implementing object characteristic graph;

[0042] When the first information acquisition module determines at least one pair of sample object pairs and the first information of each sample object in each sample object pair based on the feature graph, the first information acquisition module may be used for at least one of the following:

[0043] Calculate the behavior bearing object feature graph based on the graph neural network algorithm, determine the behavior bearing object feature of the sample object corresponding to each node in the behavior bearing object feature graph, and determine the behavior bearing object feature of each sample object in each sample object pair based on the behavior bearing object feature corresponding to each node in the behavior bearing object feature graph;

[0044] The behavior implementation object feature graph is calculated based on the graph neural network algorithm to determine the behavior implementation object features of the sample objects corresponding to each node in the behavior implementation object feature graph. Based on the behavior implementation object features corresponding to each node in the behavior implementation object feature graph, the behavior implementation object features of each sample object in each sample object pair are determined.

[0045] On the other hand, the embodiment of the present application further provides a behavior recognition device, including:

[0046] A second information acquisition module is used to acquire third information of each object to be identified in the pair of objects to be identified, the third information including an object identification feature of the object to be identified, the object identification feature including at least one of a behavior implementation object feature or a behavior receiving object feature;

[0047] A second information determination module, for determining fourth information of the pair of objects to be identified based on third information of each object to be identified in the pair of objects to be identified, the fourth information comprising a similarity between a first object to be identified and a second object to be identified in the pair of objects to be identified, the similarity representing at least one of a probability that the first object to be identified implements an action on the second object to be identified, a probability that the first object to be identified and the second object to be identified are both objects implementing the action, a probability that the first object to be identified and the second object to be identified are both objects receiving the action, or a probability that the second object to be identified implements an action on the first object to be identified;

[0048] a behavior recognition module, used for inputting the fourth information of the pair of objects to be recognized into the behavior recognition model to obtain a target behavior recognition result obtained by the behavior recognition model based on the fourth information of the pair of objects to be recognized, wherein the target behavior recognition result indicates whether there is a behavior implementation relationship between the first object to be recognized and the second object to be recognized in the pair of objects to be recognized;

[0049] Among them, the behavior recognition model is obtained by training the device based on any embodiment of the present application.

[0050] Optionally, the preset model is obtained by training the second information of the sample object pair and the first information of each sample object in the sample object pair as inputs of the preset model;

[0051] The third information also includes object characteristics and behavior characteristics of the object to be identified;

[0052] When the behavior recognition module inputs the fourth information of the object pair to be recognized into the behavior recognition model, it can be used to:

[0053] The fourth information of the object pair to be identified and the third information of each object to be identified in the object pair to be identified are input into the behavior recognition model to obtain the target behavior recognition result obtained by the behavior recognition model based on the fourth information of the object pair to be identified and the third information of each object to be identified in the object pair to be identified.

[0054] On the other hand, an embodiment of the present application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method of any embodiment of the present application.

[0055] On the other hand, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method of any embodiment of the present application are implemented.

[0056] On the other hand, an embodiment of the present application further provides a computer program product, including a computer program, which implements the steps of the method of any embodiment of the present application when the computer program is executed by a processor.

[0057] The technical solution of this embodiment is to obtain the first information of each sample object in each sample object pair, the first information includes the object identification feature of the sample object, and the object identification feature includes at least one of the behavior implementation object feature or the behavior bearing object feature; for each sample object pair, based on the first information of each sample object in the sample object pair, determine the second information of the sample object pair, the second information includes the similarity between the first sample object and the second sample object in the sample object pair; train the preset model based on the second information of each sample object pair until the training end condition is met, and obtain the behavior recognition model; because the similarity is used as the feature to characterize the sample object pair, the interpretability of the model training can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0059] Figure 1 A schematic diagram of an implementation environment for a method for training a behavior recognition model provided in an embodiment of the present application;

[0060] Figure 2 A flowchart of a method for training a behavior recognition model provided in an embodiment of the present application;

[0061] Figure 3 A schematic diagram of a behavior bearing object characteristic diagram provided in an embodiment of the present application;

[0062] Figure 4 A schematic diagram of a behavior implementation object feature map provided in an embodiment of the present application;

[0063] Figure 5 A schematic diagram of an input feature for training a preset model provided in an embodiment of the present application;

[0064] Figure 6 A flowchart of a behavior recognition method provided in an embodiment of the present application;

[0065] Figure 7 A flowchart of training a bad behavior recognition model and bad behavior recognition provided in an embodiment of the present application;

[0066] Figure 8 A schematic diagram of the structure of a training device for a behavior recognition model provided in an embodiment of the present application;

[0067] Fig. 9 A schematic diagram of the structure of a behavior recognition device provided in an embodiment of the present application;

[0068] Fig.10A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions of the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0070] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "an", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field, etc. It should be understood that when an element is "connected" or "coupled" to another element, the element may be directly connected or coupled to the other element, or may refer to the element and the other element establishing a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, such as "A and / or B" indicates implementation as "A", or implementation as "A", or implementation as "A and B".

[0071] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0072] First, several terms involved in this application are introduced and explained:

[0073] Graph embedding: The process of mapping nodes or edges in a graph into a low-dimensional vector space. These low-dimensional vectors can be used to represent the characteristics of nodes or edges. Compared with the original graph, these low-dimensional vectors are easier to process and analyze.

[0074] GraphSAGE: A graph neural network-based node representation learning method for embedding nodes in graphs or networks. The GraphSAGE model can process samples with and without label information at the same time, and can also be used to process node embedding representations in undirected graphs and partially directed graphs.

[0075] Cosine similarity: Cosine similarity is a similarity measure used to calculate the cosine of the angle between two vectors. It can be used to compare the similarity of the direction and size of two vectors.

[0076] In the related art, using the object features and the object features to train the model is a black box. It is difficult to determine the features for model training, difficult to explain the collinearity and classification results of each feature, and lacks interpretability.

[0077] In response to at least one of the above-mentioned technical problems or areas that need improvement in the related art, the present application proposes a training method for a behavior recognition model, a behavior recognition method and related devices. The scheme determines the similarity between two sample objects in a sample object pair based on at least one of the behavior object characteristics or behavior tolerance characteristics of each sample object in each sample object pair, and then uses the similarity as a feature of the training model, thereby improving the interpretability of model training.

[0078] The following describes several example implementations to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following implementations can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different implementations will not be described repeatedly.

[0079] Optionally, the present application may involve artificial intelligence (AI) technology.

[0080] Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0081] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0082] It should be noted that in the optional embodiments of the present application, the object information (object data) and other related data involved, when the embodiments in the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained through the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained. The embodiments also need to be implemented with the authorization and consent of the object.

[0083] See also Figure 1 , Figure 1 A schematic diagram of an implementation environment for a method for training a behavior recognition model provided in an embodiment of the present application.

[0084] The terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process, such as storing first information and second information. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers 104.

[0085] In this embodiment, the behavior recognition model can be trained through the server 104, and then deployed to the terminal 102. Alternatively, the behavior recognition model can be trained through the server 104, and then deployed to the server 104, and when the behavior recognition is performed, the behavior recognition is realized through the interaction between the terminal 102 and the server 104. In addition, the model can be trained and the trained model can be deployed through the terminal 102, which can be set as needed and is not limited here.

[0086] The terminal 102 may be, but is not limited to, at least one of various personal computers, laptops, smart phones, tablet computers, IoT devices, or portable wearable devices. The IoT device may be at least one of a smart speaker, a smart TV, a smart air conditioner, or a smart vehicle-mounted device. The portable wearable device may be at least one of a smart watch, a smart bracelet, or a head-mounted device. The server 104 may be implemented as an independent server 104 or a server 104 cluster consisting of multiple servers 104.

[0087] See also Figure 2 , Figure 2 A flow chart of a method for training a behavior recognition model provided in an embodiment of the present application. Figure 2The method shown in the figure may be executed by an electronic device, which may include at least one of a terminal or a server. Figure 2 The methods shown may include:

[0088] S210: Acquire first information of each sample object in each sample object pair, where the first information includes an object identification feature of the sample object, and the object identification feature includes at least one of a behavior implementing object feature or a behavior bearing object feature.

[0089] Among them, a sample object pair may include two sample objects. The object identification feature may refer to a feature that characterizes the identity of the sample object. In this embodiment, the object identification feature includes at least one of the behavior implementation object feature or the behavior bearing object feature. The behavior implementation object feature may refer to a feature that characterizes the sample object as the behavior implementation object, and may characterize the probability that the sample object is the behavior implementation object. The behavior implementation object may be understood as the party that implements the behavior. The behavior bearing object feature may be a feature that characterizes the sample object as the behavior bearing object, and may characterize the probability that the sample object is the behavior bearing object. The behavior bearing object may also be called the behavior implemented object, and may be understood as the party on whom the behavior is implemented.

[0090] In this embodiment, the sample object may include a sample account, for example, an account used to log in to an application.

[0091] Specifically, the behavior of this embodiment may refer to a behavior involving at least two parties. For example, the behavior of this embodiment may be an employment behavior, then the object of the behavior implementation may be the employer, and the object of the behavior may be the employed party. For another example, the behavior of this embodiment may be the implementation of bad behavior, such as fraud. Then the object of the behavior implementation may be the party that implements the bad behavior, such as the fraudulent party; the object of the behavior may be the party that is implemented with the bad behavior, such as the fraudulent party; the object of the behavior may be the party that is implemented with the bad behavior, such as the fraudulent party.

[0092] It is understandable that the behavior of this embodiment can be determined according to specific circumstances and is not limited here.

[0093] S220: For each sample object pair, determine second information of the sample object pair based on first information of each sample object in the sample object pair, where the second information includes a similarity between a first sample object and a second sample object in the sample object pair.

[0094] The similarity may refer to the degree of similarity between two sample objects in a sample object pair. In this embodiment, the similarity represents at least one of the probability that the first sample object implements a behavior on the second sample object, the probability that the first sample object and the second sample object are both behavior implementation objects, the probability that the first sample object and the second sample object are both behavior receiving objects, or the probability that the second sample object implements a behavior on the first sample object. Optionally, in this embodiment, the similarity may be the cosine similarity between the object identification feature of the first sample object and the object identification feature of the second sample object in the sample object pair.

[0095] S230: Train a preset model based on the second information of each sample object pair until a training end condition is met to obtain a behavior recognition model.

[0096] Among them, the preset model can be a constructed, pre-trained or non-pre-trained model. The preset model is trained based on the second information of each sample object pair, and the similarity can be used as the input of the preset model for training. The training end condition can refer to the condition for judging whether the preset model has been trained. The preset model can be a pre-training model (Pre-training model), also known as a cornerstone model or a large model, which refers to a deep neural network (DNN) with large parameters. It is trained on massive unlabeled data, and the function approximation ability of the large-parameter DNN is used to enable PTM to extract common features from the data. After fine tuning, parameter efficient fine tuning (PEFT), prompt-tuning and other technologies, it is suitable for downstream tasks. Therefore, the pre-trained model can achieve ideal results in few-shot or zero-shot scenarios. According to the data modality processed, PTM can be divided into language models (ELMO, BERT, GPT), visual models (swin-transformer, ViT, V-MOE), speech models (VALL-E), multimodal models (ViBERT, CLIP, Flamingo, Gato), etc., among which multimodal models refer to models that establish feature representations of two or more data modalities. Pre-trained models are important tools for outputting artificial intelligence generated content (AIGC), and can also be used as a general interface to connect multiple specific task models.

[0097] Optionally, each pair of sample objects may be associated with a sample label, which is a first sample label, a second sample label, a third sample label or a fourth sample label. The first sample label indicates that the two sample objects in the sample object pair are both behavior implementation objects, the second sample label indicates that the first sample object implements a behavior to the second sample object, the third sample label indicates that the second sample object implements a behavior to the first sample object, and the fourth sample label indicates that the two sample objects in the sample object pair are both behavior receiving objects. The prediction result determined by the prediction model based on the similarity is obtained, and then the prediction loss is calculated based on the prediction result and the sample label. If the prediction loss is less than the loss threshold, or the prediction loss tends to be stable, it means that the training end condition is met and the behavior recognition model is obtained.

[0098] The behavior recognition model is used to determine the target behavior recognition result of the pair of objects to be recognized, and the target behavior recognition result represents whether there is a behavior implementation relationship between the first object to be recognized and the second object to be recognized in the pair of objects to be recognized.

[0099] The technical solution of this embodiment is to obtain the first information of each sample object in each sample object pair, the first information includes the object identification feature of the sample object, and the object identification feature includes at least one of the behavior implementation object feature or the behavior bearing object feature; for each sample object pair, based on the first information of each sample object in the sample object pair, determine the second information of the sample object pair, the second information includes the similarity between the first sample object and the second sample object in the sample object pair; train the preset model based on the second information of each sample object pair until the training end condition is met, and obtain the behavior recognition model; because the similarity is used as the feature to characterize the sample object pair, the interpretability of the model training can be improved.

[0100] In a possible implementation manner, obtaining first information of each sample object in each sample object pair includes:

[0101] Obtaining sample object data of each sample object among a plurality of sample objects;

[0102] A feature graph is constructed based on the sample object data of each sample object, wherein the feature graph includes nodes corresponding to each sample object and connection relationships between the nodes, wherein the nodes in the feature graph store relevant features of the corresponding sample objects, and the connected nodes have association relationships, and the association relationships include friend relationships and behavior implementation relationships. The nodes in the feature graph that have behavior implementation relationships are pointed to behavior bearing object nodes by behavior implementation object nodes, and the behavior implementation object nodes also store object labels that characterize the sample object as a behavior implementation object, and the behavior bearing object nodes also store object labels that characterize the sample object as a behavior bearing object;

[0103] Determine at least one pair of sample object pairs and the first information of each sample object in each sample object pair based on the feature map, where the sample objects corresponding to the connected nodes in the feature map are used as a pair of sample object pairs.

[0104] In this embodiment, the sample object data of each sample object among multiple sample objects can be extracted from actual cases. The sample object data can include at least one of sample object features or behavior anomaly features. The sample object features can reflect the characteristics of the sample object. And the behavior anomaly features can reflect some abnormal operations performed by the sample object.

[0105] Optionally, the feature map can include at least one of a behavior recipient object feature map or a behavior executor object feature map. Based on the behavior recipient object feature map, the behavior recipient object features of the objects corresponding to the nodes in the behavior recipient object feature map can be determined. Based on the behavior executor object feature map, the behavior executor object features of the objects corresponding to the nodes in the behavior executor object feature map can be determined. The relevant features stored in the nodes of the behavior recipient object feature map include sample object features, and the relevant features stored in the nodes of the behavior executor object feature map include abnormal behavior features.

[0106] It should be noted that if it is necessary to determine the probability that the first sample object and the second sample object are both behavior executor objects, then a behavior executor object feature map needs to be constructed; if it is necessary to determine the probability that the first sample object and the second sample object are both behavior recipient objects, then a behavior recipient object feature map needs to be constructed; if it is necessary to determine at least one of the probability that the first sample object performs an action on the second sample object or the probability that the second sample object performs an action on the first sample object, then a behavior executor object feature map and a behavior recipient object feature map need to be constructed.

[0107] Optionally, the GraphSAGE algorithm can be used to calculate at least one of the behavior executor object features or the behavior recipient object features.

[0108] In a possible implementation, the sample object data includes at least one of first sample object data or second sample object data, the first sample object data includes sample object features, and the second sample object data includes behavior anomaly features;

[0109] Constructing a feature map based on the sample object data of each sample object includes at least one of the following:

[0110] Construct a behavior recipient object feature map based on the first sample object data of each sample object, and the nodes in the behavior recipient object feature map store the sample object features of the corresponding sample objects;

[0111] A behavior implementation object feature graph is constructed based on the second sample object data of each sample object, and the nodes in the behavior implementation object feature graph store the behavior abnormal features of the corresponding sample objects.

[0112] In this embodiment, a behavior-bearing object characteristic graph may be constructed based on the first sample object data of each sample object, and / or a behavior-implementing object characteristic graph may be constructed based on the second sample object data of each sample object.

[0113] In a possible implementation, the feature graph includes at least one of a behavior-bearing object feature graph or a behavior-implementing object feature graph;

[0114] Determining at least one pair of sample object pairs and first information of each sample object in each sample object pair based on the feature graph includes at least one of the following:

[0115] Calculate the behavior bearing object feature graph based on the graph neural network algorithm, determine the behavior bearing object feature of the sample object corresponding to each node in the behavior bearing object feature graph, and determine the behavior bearing object feature of each sample object in each sample object pair based on the behavior bearing object feature corresponding to each node in the behavior bearing object feature graph;

[0116] The behavior implementation object feature graph is calculated based on the graph neural network algorithm to determine the behavior implementation object features of the sample objects corresponding to each node in the behavior implementation object feature graph. Based on the behavior implementation object features corresponding to each node in the behavior implementation object feature graph, the behavior implementation object features of each sample object in each sample object pair are determined.

[0117] Optionally, the graph neural network algorithm may include the GraphSAGE algorithm. Specifically, GraphSAGE, or GraphSampling and Aggregation, is a node embedding learning algorithm for graph neural networks. GraphSAGE can learn a vector representation of a node from the entire graph, and this vector representation can be used for tasks such as node classification and link prediction. The main feature of GraphSAGE is that it is designed for large-scale graph pixel tasks and is a scalable algorithm. The core idea of ​​GraphSAGE: In order to perform node vector embedding in large-scale graph pixel tasks, the embedding aggregation function containing the node information should be calculated on the subgraph around each node.

[0118] In one possible implementation, the alignment is performed between a behavior implementing object characteristic map and a behavior bearing characteristic time of the object characteristic map.

[0119] In this embodiment, the structure of the training sample and the graph are the same, and the only difference is the features on the nodes. As long as the time of feature acquisition is constrained, the training of the two graph embeddings can be aligned. Specifically, since the structure of the training sample and the graph are the same, in this embodiment, the time of one of the features between the behavior implementation object feature graph and the behavior bearing object feature graph can be aligned, that is, the feature time of the implementation object feature graph and the behavior bearing object feature graph can be aligned, and there is no need to align each feature separately.

[0120] Please refer to Figure 3 and Figure 4 , Figure 3 A schematic diagram of a behavior-bearing object characteristic diagram provided in an embodiment of the present application. Figure 4 A schematic diagram of a behavior implementation object feature graph provided in an embodiment of the present application.

[0121] For Figure 3 In the behavior subject feature graph shown, the sample object features are stored on the nodes. If the case has been verified, the corresponding behavior implementation object and behavior subject will have corresponding labels, and other nodes will be unlabeled. Optionally, the GraphSAGE algorithm is used to calculate the graph embedding of all accounts as behavior subject accounts, and the graph embedding is determined as the behavior subject feature.

[0122] For Figure 4 In the behavior implementation object feature graph shown, suspicious features are stored on the nodes, such as some abnormal behavior features. If the case has been verified, the corresponding behavior implementation object and behavior subject will have corresponding labels, and other nodes will be unlabeled. Optionally, the GraphSAGE algorithm is used to calculate the graph embedding of all accounts as behavior implementation objects, and the graph embedding is determined as the behavior implementation object feature.

[0123] A node with a value of "0" indicates that it is a node corresponding to a behavior subject, and a node with a value of "1" indicates that it is a node corresponding to a behavior implementation subject. A node that is neither "0" nor "1" indicates that it has not yet been determined whether it is a behavior subject or a behavior implementation subject.

[0124] In a possible implementation manner, determining the second information of the sample object pair based on the first information of each sample object in the sample object pair includes at least one of the following:

[0125] Determine a first similarity between a behavior-performing object feature of the first sample object and a behavior-bearing object feature of the second sample object, the first similarity representing a probability that the first sample object performs a behavior on the second sample object;

[0126] Determine a second similarity between the behavior implementation object feature of the first sample object and the behavior implementation object feature of the second sample object, the second similarity representing the probability that the first sample object and the second sample object are both the behavior implementation objects;

[0127] Determine a third similarity between the behavior bearing object feature of the first sample object and the behavior bearing object feature of the second sample object, the third similarity representing the probability that the first sample object and the second sample object are both behavior bearing objects;

[0128] A fourth similarity between the behavior-bearing object feature of the first sample object and the behavior-implementing object feature of the second sample object is determined, the fourth similarity representing the probability that the second sample object implements the behavior toward the first sample object.

[0129] Optionally, the second information may include a first similarity, a second similarity, a third similarity and a fourth similarity, that is, the second information includes similarity information in multiple dimensions, which can further improve the interpretability of model training.

[0130] Please refer to Table 1, which is a schematic diagram of the similarity interpretation degree between sample object pairs provided in an embodiment of the present application.

[0131] For each account, we calculated its embedding as the object of the behavior implementation and the embedding as the object of the behavior acceptance. There are 4 embeddings for the two objects in total. The similarity between them has a certain degree of explanation:

[0132] Table 1

[0133]

[0134] In a possible implementation, the method further includes:

[0135] Obtaining an object label of each sample object in each sample object pair, the object label indicating whether the sample object is a behavior implementing object or a behavior receiving object;

[0136] Training a preset model based on the second information of each sample object pair includes:

[0137] Using the second information of the sample object pair as an input of a preset model, so that the preset model obtains a first identity prediction result of a first sample object and a second identity prediction result of a second sample object in the sample object pair based on the second information of the sample object pair;

[0138] Determining a first prediction loss between a first identity prediction result and an object label of a first sample object, and determining a second prediction loss between a second identity prediction result and an object label of a second sample object;

[0139] If it is determined that the training end condition is not met based on the first prediction loss and the second prediction loss, update the model parameters of the preset model and continue to train the preset model;

[0140] If it is determined that the training end condition is met based on the first prediction loss and the second prediction loss, end the training and use the preset model at the end of training as the behavior recognition model.

[0141] Among them, the first identity prediction result can represent whether the first sample object is the behavior executor or the behavior recipient. The second identity prediction result can represent whether the second sample object is the behavior executor or the behavior recipient.

[0142] In this embodiment, if it is determined that the training end condition is not met based on the first prediction loss and the second prediction loss, it means that the training accuracy is not enough. Therefore, after updating the model parameters of the preset model, continue to train the preset model until the preset model meets the training end condition.

[0143] Optionally, the preset model of this embodiment can be a classification model, for example, it can be an XGBoost classification model. XGBoost is also a kind of gradient boosting tree model. It also serially generates models and takes the sum of all models as the output. XGBoost makes a second-order Taylor expansion of the loss function, uses the second derivative information of the loss function to optimize the loss function, and greedily selects whether to split nodes according to whether the loss function decreases. At the same time, XGBoost adds means such as regularization, learning rate, column sampling, and approximate optimal splitting points in preventing overfitting. It also makes certain optimizations in dealing with missing values.

[0144] In a possible implementation manner, if the object label of the sample object represents that the sample object is the behavior executor, the first information of the sample object further includes the behavior abnormal feature of the sample object; if the object label of the sample object represents that the sample object is the behavior recipient, the first information of the sample object further includes the object feature of the sample object;

[0145] Using the second information of the sample object pair as the input of the preset model, so that the preset model obtains the first identity prediction result of the first sample object and the second identity prediction result of the second sample object in the sample object pair based on the second information of the sample object pair, includes:

[0146] Using the second information of the sample object pair and the first information of each sample object in the sample object pair as the input of the preset model, so that the preset model obtains the first identity prediction result and the second identity prediction result based on the second information of the sample object pair and the first information of each sample object in the sample object pair.

[0147] The technical solution of this embodiment can improve the prediction accuracy of the classification model by taking the second information of the sample object pair and the first information of each sample object in the sample object pair as inputs of the preset model.

[0148] See also Figure 5 , Figure 5 A schematic diagram of input features for training a preset model provided in an embodiment of the present application.

[0149] like Figure 5 As shown, features such as the first similarity, the second similarity, the third similarity, the fourth similarity, the sample object feature of the first sample object, the sample object feature of the second sample object, the abnormal behavior feature of the first sample object, and the abnormal behavior feature of the second sample object can be input into the XGBoost model for training.

[0150] The following embodiments illustrate how to perform behavior recognition based on any of the above embodiments.

[0151] See also Figure 6 , Figure 6 A flowchart of a behavior recognition method provided in an embodiment of the present application.

[0152] S610: Acquire third information of each object to be identified in the pair of objects to be identified, where the third information includes object identification features of the object to be identified, and the object identification features include at least one of a behavior-implementing object feature or a behavior-bearing object feature.

[0153] In this embodiment, the feature map may be updated based on the object data of each object to be identified, and then the object identification features of the object to be identified may be calculated based on the updated feature map.

[0154] S620: Determine fourth information of the pair of objects to be identified based on the third information of each object to be identified in the pair of objects to be identified, where the fourth information includes a similarity between a first object to be identified and a second object to be identified in the pair of objects to be identified.

[0155] Among them, the similarity represents at least one of the probability that the first object to be identified commits an action against the second object to be identified, the probability that the first object to be identified and the second object to be identified are both the objects of the action, the probability that the first object to be identified and the second object to be identified are both the objects of the action, or the probability that the second object to be identified commits an action against the first object to be identified.

[0156] S630: Input the fourth information of the pair of objects to be identified into the behavior recognition model to obtain a target behavior recognition result obtained by the behavior recognition model based on the fourth information of the pair of objects to be identified.

[0157] The target behavior recognition result characterizes whether there is a behavior implementation relationship between the first object to be identified and the second object to be identified in the pair of objects to be identified. Specifically, the behavior recognition model may determine the third identity prediction result of the first object to be identified based on the fourth information of the first object to be identified, and determine the fourth identity prediction result of the second object to be identified based on the fourth information of the second object to be identified. If the third identity prediction result indicates that the first object to be identified is an object for implementing the behavior, and the fourth identity prediction result indicates that the second object to be identified is an object for bearing the behavior, then it is determined that the first object to be identified implements the behavior to the second object to be identified; if the third identity prediction result indicates that the first object to be identified is an object for bearing the behavior, and the fourth identity prediction result indicates that the second object to be identified is an object for implementing the behavior, then it is determined that the second object to be identified implements the behavior to the first object to be identified; if the third identity prediction result indicates that the first object to be identified is an object for implementing the behavior, and the fourth identity prediction result indicates that the second object to be identified is an object for implementing the behavior, then the first object to be identified and the second object to be identified are both objects for implementing the behavior; if the third identity prediction result indicates that the first object to be identified is an object for bearing the behavior, and the fourth identity prediction result indicates that the second object to be identified is an object for bearing the behavior, then the first object to be identified and the second object to be identified are both objects for bearing the behavior; if at least one of the following conditions is satisfied: the third identity prediction result indicates that the first object to be identified is neither an object for implementing the behavior nor an object for bearing the behavior, or the fourth identity prediction result indicates that the second object to be identified is neither an object for implementing the behavior nor an object for bearing the behavior, then it is determined that there is no behavior implementation relationship between the first object to be identified and the second object to be identified.

[0158] The behavior recognition model of this embodiment can be trained in the manner described in any of the above embodiments, and is not limited here.

[0159] In the technical solution of this embodiment, since the behavior recognition model is trained based on the method described in any of the above embodiments, the behavior recognition model has a good degree of interpretability. At the same time, it can automatically identify whether there is a behavior implementation relationship between the first object to be identified and the second object to be identified based on the behavior recognition model, which can improve the efficiency and interpretability of behavior recognition.

[0160] In a possible implementation manner, the preset model is obtained by training the second information of the sample object pair and the first information of each sample object in the sample object pair as inputs of the preset model;

[0161] The third information also includes object characteristics and behavior characteristics of the object to be identified;

[0162] Inputting the fourth information of the pair of objects to be identified into the behavior recognition model includes:

[0163] The fourth information of the object pair to be identified and the third information of each object to be identified in the object pair to be identified are input into the behavior recognition model to obtain the target behavior recognition result obtained by the behavior recognition model based on the fourth information of the object pair to be identified and the third information of each object to be identified in the object pair to be identified.

[0164] The technical solution of this embodiment can improve the accuracy of behavior recognition by inputting the fourth information of the object pair to be identified and the third information of each object to be identified in the object pair to be identified into the behavior recognition model to obtain the target behavior recognition result obtained by the behavior recognition model based on the fourth information of the object pair to be identified and the third information of each object to be identified in the object pair to be identified.

[0165] For ease of understanding, the following embodiments are based on any of the above embodiments and take the implementation of bad behavior as an example for illustration.

[0166] See also Figure 7 , Figure 7 A flowchart of training a bad behavior recognition model and bad behavior recognition is provided in an embodiment of the present application.

[0167] In general, we collect existing bad behavior implementation cases and all objects to build two directed graphs. If it is a verified bad behavior implementation case, the edge is from the bad behavior implementation object to the bad behavior receiving object, otherwise only the friendship part is built. Then build a graph based on the suspicious features of bad behavior implementation and train the embedding of bad behavior implementers. Build a graph based on the features of bad behavior receiving, and calculate the graph embedding of all objects as potential bad behavior receiving accounts. For each account, they have an embedding as a bad behavior implementation account and an embedding as a potential bad behavior receiving account. The four embeddings of "bad behavior implementation account-deceived account" are similar to each other, and four features are obtained. Add the original bad behavior implementation features (abnormal behavior features) and bad behavior receiving features (sample object features). Based on the labels of bad behavior implementation cases, XGBoost is used to train a classification model, and then deployed to the real-time module for real-time detection of bad behavior implementation.

[0168] The specific implementation process can be as follows:

[0169] 1. Construct two directed graphs based on the existing bad behavior cases and all objects. If it is a verified bad behavior case, the edge is from the bad behavior implementation object to the bad behavior recipient object. Otherwise, only the friendship parts are constructed.

[0170] 2. For the first graph, construct edges in the same way as described in step 1. The nodes store the potential bad behavior characteristics. If there are verified bad behavior cases, the corresponding bad behavior implementation objects and bad behavior tolerance objects will have corresponding labels, and other nodes will be unlabeled. Use the GraphSAGE algorithm to calculate the graph embedding of all accounts as bad behavior tolerance accounts;

[0171] 3. For the second graph, the same method as step 1 is used to construct edges. The difference is that the nodes store suspicious features for the bad behavior implementation model, such as some abnormal behavior features. If the bad behavior implementation case has been verified, the corresponding bad behavior implementation object and bad behavior recipient will have corresponding labels, and other nodes will be unlabeled. The GraphSAGE algorithm is used to calculate the graph embedding of all accounts as bad behavior implementation objects.

[0172] 4. For each account, they have an embedding of the account that performs bad behavior and an embedding of the account that is a potential bearer of bad behavior. The similarity between these two embeddings is calculated to obtain four similarity features. The cosine similarity is used to calculate the similarity between vectors.

[0173] 5. Combine the original bad behavior implementation characteristics and bad behavior tolerance characteristics, plus the similarity characteristics obtained in step 4, and use XGBoost to train a classification model based on bad behavior implementation cases. Then deploy it to the real-time module for real-time detection of bad behavior implementation.

[0174] Specifically, the technical solution of the embodiment of the present application can be applied to the business of anti-bad behavior implementation. In the implementation of bad behavior, in order to improve the success rate of bad behavior implementation, the object of bad behavior implementation will circle a part of the people who are more likely to bear bad behavior to cheat. In addition, at this stage, the object of bad behavior implementation will use normal objects to assist, and these objects may be bad behavior implementation numbers or bad behavior acceptance numbers at the same time. It is observed that as bad behavior implementation numbers, their malicious patterns will have some similarities, and as bad behavior acceptance objects, there is also a certain degree of similarity. At the same time, if this relationship pair is considered at the same time, there can be a certain degree of aggregation under the same pattern. Therefore, this scheme uses embedding to characterize their embeddings as bad behavior implementation accounts and as bad behavior recipients, and then characterizes the characteristics of this relationship pair through the similarity between the two embeddings. Finally, it is input into the XGBoost classification model. It can not only use graph embedding to characterize the structural characteristics of the object, but also take into account the problem of the degree of interpretability after the introduction of embedding, which has a significant improvement effect on the existing XGBoost model.

[0175] It should be noted that for some scenarios where the degree of explainability is not very important, the graph classification algorithm can be directly used to perform classification in one step, but the technical problems of this method are also obvious:

[0176] 1. Poor interpretability. Graph algorithms are a black box for external use, and it is difficult to explain the contribution of each feature and the classification results.

[0177] 2. Relying on a large number of samples. The number of samples that the graph classification algorithm relies on is often much larger than the number of samples that the decision tree model relies on;

[0178] 3. Poor real-time performance. Currently, if graph classification algorithms are to be applied to real-time scenarios, they are usually implemented through offline prediction and online reading. This method is still an order of magnitude less real-time than real-time prediction.

[0179] The technical solution provided in the embodiments of the present application can solve the above technical problems.

[0180] See also Figure 8 , Figure 8 A schematic diagram of a training device for a behavior recognition model provided in an embodiment of the present application. Figure 8 The device 800 shown may include a first information acquisition module 810, a first information determination module 820 and a training module 830, wherein:

[0181] A first information acquisition module 810 is used to acquire first information of each sample object in each sample object pair, wherein the first information includes an object identification feature of the sample object, and the object identification feature includes at least one of a behavior implementation object feature or a behavior acceptance object feature;

[0182] A first information determination module 820 is configured to determine, for each sample object pair, second information of the sample object pair based on the first information of each sample object in the sample object pair, wherein the second information includes a similarity between a first sample object and a second sample object in the sample object pair, wherein the similarity represents at least one of a probability that the first sample object performs a behavior on the second sample object, a probability that the first sample object and the second sample object are both behavior-performing objects, a probability that the first sample object and the second sample object are both behavior-bearing objects, or a probability that the second sample object performs a behavior on the first sample object;

[0183] The training module 830 is used to train a preset model based on the second information of each sample object pair until the training end condition is met to obtain a behavior recognition model. The behavior recognition model is used to determine the identity recognition result of the object to be recognized. The identity recognition result includes the object that implements the behavior or the object that receives the behavior.

[0184] Optionally, the device is further used to obtain an object label of each sample object in each sample object pair, the object label representing that the sample object is a behavior implementation object or a behavior receiving object;

[0185] When training the preset model based on the second information of each sample object pair, the training module 830 can be used to:

[0186] Using the second information of the sample object pair as an input of a preset model, so that the preset model obtains a first identity prediction result of a first sample object and a second identity prediction result of a second sample object in the sample object pair based on the second information of the sample object pair;

[0187] Determining a first prediction loss between a first identity prediction result and an object label of a first sample object, and determining a second prediction loss between a second identity prediction result and an object label of a second sample object;

[0188] If it is determined based on the first prediction loss and the second prediction loss that the training end condition is not met, updating the model parameters of the preset model and continuing to train the preset model;

[0189] If it is determined based on the first prediction loss and the second prediction loss that the training end condition is met, the training is ended, and the preset model after the training is completed is used as the behavior recognition model.

[0190] Optionally, if the object label of the sample object indicates that the sample object is a behavior-performing object, the first information of the sample object further includes the abnormal behavior characteristics of the sample object; if the object label of the sample object indicates that the sample object is a behavior-bearing object, the first information of the sample object further includes the object characteristics of the sample object;

[0191] When the training module 830 uses the second information of the sample object pair as the input of the preset model so that the preset model obtains the first identity prediction result of the first sample object and the second identity prediction result of the second sample object in the sample object pair based on the second information of the sample object pair, it can be used to:

[0192] The second information of the sample object pair and the first information of each sample object in the sample object pair are used as inputs of a preset model, so that the preset model obtains a first identity prediction result and a second identity prediction result based on the second information of the sample object pair and the first information of each sample object in the sample object pair.

[0193] Optionally, when determining the second information of the sample object pair based on the first information of each sample object in the sample object pair, the first information determining module 820 may be used for at least one of the following:

[0194] Determine a first similarity between a behavior-performing object feature of the first sample object and a behavior-bearing object feature of the second sample object, the first similarity representing a probability that the first sample object performs a behavior on the second sample object;

[0195] Determine a second similarity between the behavior implementation object feature of the first sample object and the behavior implementation object feature of the second sample object, the second similarity representing the probability that the first sample object and the second sample object are both the behavior implementation objects;

[0196] Determine a third similarity between the behavior bearing object feature of the first sample object and the behavior bearing object feature of the second sample object, the third similarity representing the probability that the first sample object and the second sample object are both behavior bearing objects;

[0197] A fourth similarity between the behavior-bearing object feature of the first sample object and the behavior-implementing object feature of the second sample object is determined, the fourth similarity representing the probability that the second sample object implements the behavior toward the first sample object.

[0198] Optionally, when acquiring the first information of each sample object in each sample object pair, the first information acquisition module 810 may be used to:

[0199] Obtaining sample object data of each sample object among a plurality of sample objects;

[0200] A feature graph is constructed based on the sample object data of each sample object, wherein the feature graph includes nodes corresponding to each sample object and connection relationships between the nodes, wherein the nodes in the feature graph store relevant features of the corresponding sample objects, and the connected nodes have association relationships, and the association relationships include friend relationships and behavior implementation relationships. The nodes in the feature graph that have behavior implementation relationships are pointed to behavior bearing object nodes by behavior implementation object nodes, and the behavior implementation object nodes also store object labels that characterize the sample object as a behavior implementation object, and the behavior bearing object nodes also store object labels that characterize the sample object as a behavior bearing object;

[0201] At least one pair of sample object pairs and first information of each sample object in each sample object pair are determined based on the feature graph, wherein the sample objects corresponding to the nodes connected in the feature graph are taken as a pair of sample object pairs.

[0202] Optionally, the sample object data includes at least one of first sample object data or second sample object data, the first sample object data includes sample object features, and the second sample object data includes abnormal behavior features;

[0203] When constructing a feature map based on the sample object data of each sample object, the first information acquisition module 810 may be used for at least one of the following:

[0204] Building a behavior bearing object feature graph based on the first sample object data of each sample object, wherein the nodes in the behavior bearing object feature graph store sample object features of the corresponding sample object;

[0205] A behavior implementation object feature graph is constructed based on the second sample object data of each sample object, and the nodes in the behavior implementation object feature graph store the behavior abnormal features of the corresponding sample objects.

[0206] Optionally, the characteristic graph includes at least one of a behavior-bearing object characteristic graph or a behavior-implementing object characteristic graph;

[0207] When determining at least one pair of sample object pairs and first information of each sample object in each sample object pair based on the feature graph, the first information acquisition module 810 may be used for at least one of the following:

[0208] Calculate the behavior bearing object feature graph based on the graph neural network algorithm, determine the behavior bearing object feature of the sample object corresponding to each node in the behavior bearing object feature graph, and determine the behavior bearing object feature of each sample object in each sample object pair based on the behavior bearing object feature corresponding to each node in the behavior bearing object feature graph;

[0209] The behavior implementation object feature graph is calculated based on the graph neural network algorithm to determine the behavior implementation object features of the sample objects corresponding to each node in the behavior implementation object feature graph. Based on the behavior implementation object features corresponding to each node in the behavior implementation object feature graph, the behavior implementation object features of each sample object in each sample object pair are determined.

[0210] See also Fig. 9 , Fig. 9 This is a schematic diagram of the structure of a behavior recognition device provided in an embodiment of the present application. Fig. 9 The device 900 shown may include a second information acquisition module 910, a second information determination module 920 and a behavior recognition module 930, wherein:

[0211] The second information acquisition module 910 is used to acquire third information of each object to be identified in the pair of objects to be identified, the third information including object identification features of the object to be identified, the object identification features including at least one of a behavior implementation object feature or a behavior receiving object feature;

[0212] A second information determination module 920 is used to determine fourth information of the pair of objects to be identified based on the third information of each object to be identified in the pair of objects to be identified, the fourth information including the similarity between the first object to be identified and the second object to be identified in the pair of objects to be identified, the similarity representing at least one of the probability that the first object to be identified implements an action on the second object to be identified, the probability that the first object to be identified and the second object to be identified are both objects of the action, the probability that the first object to be identified and the second object to be identified are both objects of the action, or the probability that the second object to be identified implements an action on the first object to be identified;

[0213] The behavior recognition module 930 is used to input the fourth information of the pair of objects to be identified into the behavior recognition model to obtain a target behavior recognition result obtained by the behavior recognition model based on the fourth information of the pair of objects to be identified. The target behavior recognition result represents whether there is a behavior implementation relationship between the first object to be identified and the second object to be identified in the pair of objects to be identified.

[0214] Optionally, the preset model is obtained by training the second information of the sample object pair and the first information of each sample object in the sample object pair as inputs of the preset model;

[0215] The third information also includes object characteristics and behavior characteristics of the object to be identified;

[0216] When the behavior recognition module 930 inputs the fourth information of the object pair to be recognized into the behavior recognition model, it can be used to:

[0217] The fourth information of the object pair to be identified and the third information of each object to be identified in the object pair to be identified are input into the behavior recognition model to obtain the target behavior recognition result obtained by the behavior recognition model based on the fourth information of the object pair to be identified and the third information of each object to be identified in the object pair to be identified.

[0218] The device of the embodiments of the present application can execute the method provided by the embodiments of the present application, and the implementation principles are similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0219] An embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, and the processor executes the above computer program to implement the steps of the method of any embodiment of the present application.

[0220] In an alternative embodiment, an electronic device is provided, such as Fig.10 As shown, Fig.10 The electronic device 1000 shown includes: a processor 1001 and a memory 1003. The processor 1001 and the memory 1003 are connected, such as through a bus 1002. Optionally, the electronic device 1000 may also include a transceiver 1004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 1004 is not limited to one, and the structure of the electronic device 1000 does not constitute a limitation on the embodiments of the present application.

[0221] Processor 1001 may be a CPU (Central Processing Unit), a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute the various example logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 1001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0222] The bus 1002 may include a path to transmit information between the above components. The bus 1002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 1002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0223] The memory 1003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.

[0224] The memory 1003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the computer program stored in the memory 1003 to implement the steps shown in the above method embodiment.

[0225] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0226] The embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0227] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0228] The above are only optional implementation methods for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the scheme of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. A training method for a behavior recognition model, It is characterized in that include: Acquire first information of each sample object in each sample object pair, wherein the first information includes an object identification feature of the sample object, and the object identification feature includes at least one of a behavior implementation object feature or a behavior acceptance object feature; For each sample object pair, based on the first information of each sample object in the sample object pair, second information of the sample object pair is determined, the second information including the similarity between the first sample object and the second sample object in the sample object pair, the similarity representing at least one of the probability that the first sample object performs a behavior on the second sample object, the probability that the first sample object and the second sample object are both behavior-performing objects, the probability that the first sample object and the second sample object are both behavior-bearing objects, or the probability that the second sample object performs a behavior on the first sample object; A preset model is trained based on the second information of each sample object pair until the training end condition is met to obtain a behavior recognition model, wherein the behavior recognition model is used to determine a target behavior recognition result of the object pair to be identified, and the target behavior recognition result characterizes whether there is a behavior implementation relationship between the first object to be identified and the second object to be identified in the object pair to be identified.

2. The method according to claim 1, It is characterized in that The method further comprises: Obtaining an object label of each sample object in each sample object pair, wherein the object label indicates that the sample object is a behavior implementing object or a behavior receiving object; The step of training a preset model based on the second information of each sample object pair includes: Using the second information of the sample object pair as input of the preset model, so that the preset model obtains a first identity prediction result of the first sample object in the sample object pair and a second identity prediction result of the second sample object in the sample object pair based on the second information of the sample object pair; Determining a first prediction loss between the first identity prediction result and the object label of the first sample object, and determining a second prediction loss between the second identity prediction result and the object label of the second sample object; If it is determined based on the first prediction loss and the second prediction loss that the training end condition is not met, updating the model parameters of the preset model and continuing to train the preset model; If it is determined based on the first prediction loss and the second prediction loss that the training end condition is met, the training is ended, and the preset model after the training is completed is used as the behavior recognition model.

3. The method according to claim 2, It is characterized in that If the object label of the sample object indicates that the sample object is a behavior implementation object, the first information of the sample object further includes abnormal behavior characteristics of the sample object; If the object label of the sample object indicates that the sample object is a behavior-bearing object, the first information of the sample object further includes the object feature of the sample object; The using the second information of the sample object pair as the input of the preset model so that the preset model obtains a first identity prediction result of the first sample object and a second identity prediction result of the second sample object in the sample object pair based on the second information of the sample object pair, comprises: The second information of the sample object pair and the first information of each sample object in the sample object pair are used as inputs of the preset model, so that the preset model obtains the first identity prediction result and the second identity prediction result based on the second information of the sample object pair and the first information of each sample object in the sample object pair.

4. The method according to claim 1, It is characterized in that The determining the second information of the sample object pair based on the first information of each sample object in the sample object pair includes at least one of the following: Determine a first similarity between a behavior-performing object feature of the first sample object and a behavior-bearing object feature of the second sample object, wherein the first similarity represents a probability that the first sample object performs a behavior on the second sample object; Determine a second similarity between a behavior implementation object feature of the first sample object and a behavior implementation object feature of the second sample object, wherein the second similarity represents a probability that the first sample object and the second sample object are both behavior implementation objects; Determining a third similarity between the behavior bearing object feature of the first sample object and the behavior bearing object feature of the second sample object, the third similarity representing a probability that the first sample object and the second sample object are both behavior bearing objects; A fourth similarity between the behavior-bearing object feature of the first sample object and the behavior-implementing object feature of the second sample object is determined, wherein the fourth similarity represents a probability that the second sample object implements a behavior toward the first sample object.

5. The method according to claim 1, It is characterized in that The obtaining of first information of each sample object in each sample object pair includes: Obtaining sample object data of each sample object among a plurality of sample objects; Constructing a feature graph based on the sample object data of each sample object, the feature graph includes nodes corresponding to each sample object and connection relationships between the nodes, wherein the nodes in the feature graph store relevant features of the corresponding sample objects, and there is an association relationship between the connected nodes, the association relationship includes a friend relationship and a behavior implementation relationship, and the nodes in the feature graph with a behavior implementation relationship are pointed to a behavior bearing object node by a behavior implementation object node, the behavior implementation object node also stores an object label representing that the sample object is a behavior implementation object, and the behavior bearing object node also stores an object label representing that the sample object is a behavior bearing object; At least one pair of sample object pairs and first information of each sample object in each sample object pair are determined based on the feature graph, wherein the sample objects corresponding to the nodes connected in the feature graph are taken as a pair of sample object pairs.

6. The method according to claim 5, It is characterized in that The sample object data includes at least one of first sample object data or second sample object data, the first sample object data includes sample object features, and the second sample object data includes abnormal behavior features; The constructing a feature map based on the sample object data of each sample object includes at least one of the following: Building a behavior bearing object characteristic graph based on the first sample object data of each sample object, wherein the nodes in the behavior bearing object characteristic graph store sample object characteristics of the corresponding sample object; A behavior implementation object feature graph is constructed based on the second sample object data of each sample object, and the nodes in the behavior implementation object feature graph store the behavior abnormal features of the corresponding sample objects.

7. The method according to claim 5, It is characterized in that The characteristic graph includes at least one of a behavior-bearing object characteristic graph or a behavior-implementing object characteristic graph; The determining, based on the feature graph, at least one pair of sample object pairs and first information of each sample object in each sample object pair includes at least one of the following: Calculating the behavior bearing object feature graph based on a graph neural network algorithm to determine the behavior bearing object feature of the sample object corresponding to each node in the behavior bearing object feature graph, and determining the behavior bearing object feature of each sample object in each sample object pair based on the behavior bearing object feature corresponding to each node in the behavior bearing object feature graph; The behavior implementation object feature graph is calculated based on a graph neural network algorithm to determine the behavior implementation object features of the sample objects corresponding to each node in the behavior implementation object feature graph, and based on the behavior implementation object features corresponding to each node in the behavior implementation object feature graph, the behavior implementation object features of each sample object in each sample object pair are determined.

8. A behavior recognition method, It is characterized in that include: Acquire third information of each object to be identified in the pair of objects to be identified, the third information including an object identification feature of the object to be identified, the object identification feature including at least one of a behavior-performing object feature or a behavior-bearing object feature; Based on the third information of each object to be identified in the pair of objects to be identified, determining the fourth information of the pair of objects to be identified, the fourth information including the similarity between the first object to be identified and the second object to be identified in the pair of objects to be identified, the similarity representing at least one of the probability that the first object to be identified performs an action on the second object to be identified, the probability that the first object to be identified and the second object to be identified are both objects of the action, the probability that the first object to be identified and the second object to be identified are both objects of the action, or the probability that the second object to be identified performs an action on the first object to be identified; Inputting the fourth information of the pair of objects to be identified into a behavior recognition model to obtain a target behavior recognition result obtained by the behavior recognition model based on the fourth information of the pair of objects to be identified, wherein the target behavior recognition result indicates whether there is a behavior implementation relationship between the first object to be identified and the second object to be identified in the pair of objects to be identified; Wherein, the behavior recognition model is trained based on the method described in any one of claims 1-8.

9. The method according to claim 8, It is characterized in that The preset model is obtained by training by taking the second information of the sample object pair and the first information of each sample object in the sample object pair as inputs of the preset model; The third information also includes object characteristics and behavior characteristics of the object to be identified; The step of inputting the fourth information of the pair of objects to be identified into the behavior recognition model comprises: The fourth information of the pair of objects to be identified and the third information of each object to be identified in the pair of objects to be identified are input into the behavior recognition model to obtain a target behavior recognition result obtained by the behavior recognition model based on the fourth information of the pair of objects to be identified and the third information of each object to be identified in the pair of objects to be identified.

10. A training device for a behavior recognition model, It is characterized in that include: A first information acquisition module, configured to acquire first information of each sample object in each sample object pair, wherein the first information includes an object identification feature of the sample object, and the object identification feature includes at least one of a behavior implementation object feature or a behavior acceptance object feature; a first information determination module, configured to determine, for each sample object pair, second information of the sample object pair based on the first information of each sample object in the sample object pair, wherein the second information includes a similarity between a first sample object and a second sample object in the sample object pair, wherein the similarity represents at least one of a probability that the first sample object performs a behavior on the second sample object, a probability that the first sample object and the second sample object are both behavior-performing objects, a probability that the first sample object and the second sample object are both behavior-bearing objects, or a probability that the second sample object performs a behavior on the first sample object; A training module is used to train a preset model based on the second information of each sample object pair until the training end condition is met to obtain a behavior recognition model, wherein the behavior recognition model is used to determine the identity recognition result of the object to be recognized, and the identity recognition result includes the object implementing the behavior or the object receiving the behavior.

11. A behavior recognition device, It is characterized in that include: A second information acquisition module is used to acquire third information of each object to be identified in the pair of objects to be identified, wherein the third information includes an object identification feature of the object to be identified, and the object identification feature includes at least one of a behavior-performing object feature or a behavior-bearing object feature; a second information determination module, configured to determine fourth information of the pair of objects to be identified based on third information of each object to be identified in the pair of objects to be identified, wherein the fourth information includes a similarity between a first object to be identified and a second object to be identified in the pair of objects to be identified, wherein the similarity represents at least one of a probability that the first object to be identified implements an action toward the second object to be identified, a probability that the first object to be identified and the second object to be identified are both objects that implement the action, a probability that the first object to be identified and the second object to be identified are both objects that receive the action, or a probability that the second object to be identified implements an action toward the first object to be identified; a behavior recognition module, configured to input the fourth information of the pair of objects to be recognized into a behavior recognition model, so as to obtain a target behavior recognition result obtained by the behavior recognition model based on the fourth information of the pair of objects to be recognized, wherein the target behavior recognition result indicates whether there is a behavior implementation relationship between the first object to be recognized and the second object to be recognized in the pair of objects to be recognized; Wherein, the behavior recognition model is trained based on the device described in claim 10.

12. An electronic device comprising a memory, a processor and a computer program stored in the memory, It is characterized in that The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 9.

13. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

14. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images