A method for detecting key behaviors of intelligent agents based on feature fusion

Abstract

The present invention provides a method for detecting key behaviors of an intelligent agent based on feature fusion, the method comprising acquiring a multimodal training sample set, the training sample set comprising sequence data consisting of environmental image data and action text data; extracting features from the environmental image data of the intelligent agent using an image encoding module based on Inception and GRU neural network to generate an image encoding vector of fixed dimension through key judgment; extracting features from the text data of the action text of the intelligent agent t The action text encoding vector is obtained by using the ALBERT-based text encoding module; the image encoding vector and the action text encoding vector are fused and detected. The action text data of the multimodal training sample set is marked, and the key action detection is performed after re-fusion, which helps to solve the technical problem of adding experts that is lacking in the prior art.

Description

Technical field:

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method for detecting key behaviors of an intelligent agent based on feature fusion. Background technology:

[0002] In recent years, with the breakthrough progress made in artificial intelligence technology, the development and research of intelligent agent learning have been promoted. The complexity of the environment in which intelligent agents are located has also continued to increase. The behavioral decision-making model based on human expert rules has been difficult to adapt to the trend of rapid changes in the intelligent agent environment. In the future, intelligent agents urgently need fast, automatic and autonomous behavioral decisions to improve their decision-making behavior capabilities, so as to adapt to high-speed, complex and changing environments. In recent years, with the development of artificial intelligence technology, the research and breakthroughs of cutting-edge technologies such as deep learning and reinforcement learning have improved the automation and intelligence of intelligent agents to a certain extent. Some new behavioral actions have emerged in the process of behavioral decision-making deduction of artificial intelligence decision-making agents, which can in turn provide reference for the behavioral decision-making of human experts.

[0003] Since the scale of decision-making deduction of intelligent agents is often large, the training simulation time is long, and the entity data involved in the environment is huge and diverse. Under such circumstances, it is very difficult for human experts to locate key behavioral actions from the long-term and high-frequency decision-making behavior of artificial intelligence decision-making agents. It is time-consuming and there is an inaccurate problem of locating key behavioral actions. Therefore, an innovative behavioral action mining method is urgently needed to automatically and intelligently extract key environmental data and key behavioral actions made at this node from the simulation data after the training deduction, deepen human experts' understanding of the behavioral actions made by artificial intelligence algorithms, improve the efficiency of human experts in command decision-making training for intelligent agent deduction, and discover new behavioral actions to improve the level of human experts.

[0004] There is an urgent need for a method for detecting key behaviors of intelligent agents based on feature fusion, which can help solve the technical problem of adding experts that is lacking in existing technologies. Summary of the invention:

[0005] In one embodiment, the present invention provides a method for detecting key behaviors of intelligent agents based on feature fusion, by identifying the behavioral action text data in a multimodal training sample set, and performing action detection of key behaviors after re-integration, which helps to solve the technical problem of adding experts that is lacking in the prior art.

[0006] The method for detecting key behaviors of an intelligent agent based on feature fusion includes:

[0007] Acquire a multimodal training sample set, wherein the training sample set includes sequence data consisting of paired environment image data of an intelligent agent and labeled behavior action text data of the intelligent agent;

[0008] The environment image data of the intelligent agent is subjected to feature extraction using an image encoding module based on Inception and GRU neural networks to generate an image encoding vector of fixed dimension through key judgment;

[0009] Text data of the behavior of the agent t Use the ALBERT-based text encoding module to encode text features and obtain a fixed-dimensional action text encoding vector.

[0010] The image coding vector and the action text coding vector are fused and then output as categories through a fully connected layer, thereby realizing the detection of key behavioral actions.

[0011] In one embodiment, the multimodal training sample set is obtained, wherein the training sample set includes sequence data consisting of paired environment image data of an intelligent agent and labeled behavior action text data of the intelligent agent, including:

[0012] When the agent is simulating or running, the environment image data between the K time steps before and after the action time t of the agent is collected to form the environment image sequence data of the agent;

[0013] Collect the current action information as the action text dataset;

[0014] Receive training data for generating a detection model by marking the action text data set, wherein the training data for the detection model includes a pair of environment image data of an intelligent agent and behavior action text data of the intelligent agent, as well as a category label;

[0015] Repeat the above process to finally generate a model training sequence dataset.

[0016] In one embodiment, the step of extracting features of the environment image data of the agent using an image encoding module based on Inception and GRU neural network to generate an image encoding vector of fixed dimension through key judgment includes:

[0017] For the agent environment image data at each moment, Inception, which has excellent performance in image processing tasks, is used as a feature extractor to mine image frame-level features and output image frame-level sequence features;

[0018] The GRU network is used to model the dependencies between sequences. After inputting the GRU model, the output of the last layer of the GRU model is taken as the total fixed-dimensional image encoding vector.

[0019] In one embodiment, the attention mechanism is calculated using the following calculation formula:

[0020]

[0021] Where the query vector is defined as Q = x α W Q , the key vector is K = x β W K , the value vector is V = x β W V , where W Q , W K , W V is the weight matrix, d k is the dimension of the value vector K, x α and x β are feature vectors of different modes, which are image feature encoding vectors imgIn in the present invention. t And the text feature encoding vector textIn t .

[0022] In one embodiment, the binary cross entropy function:

[0023]

[0024] Define a gradient modulus as:

[0025]

[0026] The gradient modulus g represents the distance between the true value and the predicted value of the sample.

[0027] In one embodiment, the gradient density function is defined as the number of samples distributed within a unit value region in the following form:

[0028]

[0029] where g k represents the gradient of the kth sample, and

[0030]

[0031] So the gradient density function GD(g) means that the gradient falls in the region The number of samples.

[0032] In one embodiment, the GHM Loss for classification is defined as follows:

[0033]

[0034] After the entire neural network model is built and the loss function is selected:

[0035] Use {(img 1 ,text 1 ;label 1 ),...,{(img t ,text t ;label t ),...,{(img T ,text T ;label T )} dataset for model training. Description of the drawings:

[0036] Figure 1 A schematic diagram of a key behavior detection model of an intelligent agent based on feature fusion in one embodiment of the present invention;

[0037] Figure 2 A schematic diagram of an attention-based multimodal fusion neural network in another embodiment of the present invention;

[0038] Figure 3 It is a schematic diagram of the process of the present invention in another embodiment of the present invention. Specific embodiment:

[0039] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0040] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0041] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0042] It should also be understood that, although the present application has been described with reference to some specific examples, those skilled in the art will be able to realize many other equivalent forms of the present application that have the features described in the claims and are therefore within the scope of protection defined thereby.

[0043] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0044] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to determine the true intent based on the user's historical operations and to avoid unnecessary or redundant details that make the present application unclear. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but are merely used as the basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0045] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0046] Figure 1 A schematic diagram of a key behavior detection model of an intelligent agent based on feature fusion in one embodiment of the present invention; Figure 2 A schematic diagram of an attention-based multimodal fusion neural network in another embodiment of the present invention; Figure 3 It is a schematic diagram of the process of the present invention in another embodiment of the present invention.

[0047] In one embodiment, the present invention provides a method for detecting key behaviors of an intelligent agent based on feature fusion, and the method for detecting key behaviors of an intelligent agent based on feature fusion includes:

[0048] S101, obtaining a multimodal training sample set, wherein the training sample set includes sequence data consisting of paired environment image data of intelligent agents and labeled behavior action text data of the intelligent agents;

[0049] S102, extracting features from the environment image data of the agent using an image coding module based on Inception and GRU neural network to generate an image coding vector of fixed dimension through key judgment;

[0050] S103, the behavior action text data text of the intelligent agent t Use the ALBERT-based text encoding module to encode text features and obtain a fixed-dimensional action text encoding vector.

[0051] Each agent's action text data t Use the pre-trained ALBERT model to encode to obtain rich and diverse text representations and obtain a fixed-dimensional action text encoding vector textIn t .

[0052] S104, fusing the image coding vector and the action text coding vector, and then outputting the categories through a fully connected layer, thereby realizing the detection of key behavioral actions.

[0053] The present invention provides a specific implementation of a method for detecting key behaviors of an intelligent agent based on feature fusion.

[0054] In one embodiment, the multimodal training sample set is obtained, wherein the training sample set includes sequence data consisting of paired environment image data of an intelligent agent and labeled behavior action text data of the intelligent agent, including:

[0055] Specifically, when the agent is simulating or running, it collects the environmental image data between the K time steps before and after the action time t of the agent to form the environmental image sequence data (x t-K , x t-K+1 , ..., x t , ..., x t+K-1， x t+K ), denoted as img t Collect the current action information as the action text dataset, denoted as text t .

[0056] Collect the current action information as the action text dataset;

[0057] Receive the training data of the detection model generated by marking the action text data set, wherein the training data of the detection model includes a pair of environmental image data of the intelligent agent and the action text data of the intelligent agent, as well as a category label. The marking here is completed by manual operation, and the human expert manually reviews and annotates the action to generate a category label label of whether it is a critical action. t , forming a pair of agent environment image data img t and the agent's behavior text data t And the category label label t , as the training data for the key behavior detection model (img t ,text t ;label t ).

[0058] Repeat the above steps at different times to form the final model training sequence dataset:

[0059] {(img 1 ,text 1 ;label 1 ),...,{(img t,text t ;label t ),...,{(img T ,text T ;label T )};

[0060] Repeat the above process to finally generate a model training sequence dataset.

[0061] In one embodiment, the step of extracting features of the environment image data of the agent using an image encoding module based on Inception and GRU neural network to generate an image encoding vector of fixed dimension through key judgment includes:

[0062] For the agent environment image data at each moment, Inception, which has excellent performance in image processing tasks, is used as a feature extractor to mine image frame-level features and output image frame-level sequence features;

[0063] Specifically, the original image sequence data (x t-K , x t-K+1 , ..., x t , ..., x t+K-1 , x t+K ) is input into the Inception network model to obtain the image frame-level sequence feature representation output by the model (i t-K ,i t-K+1 , ..., i t , ..., i t+K-1 ,i t+K );

[0064] The judgment of key actions needs to be based on the environment before the agent's action and the changes in the environment after the action. Therefore, the image frame level sequence feature representation (i t-K ,i t-K+1 , ..., i t , ..., i t+K-1 ,i t+K ), use the GRU network to model the dependencies between sequences, input the GRU model and take the output of the last layer of the GRU model as the total fixed-dimensional image encoding vector imgIn t .

[0065] The GRU network is used to model the dependencies between sequences. After inputting the GRU model, the output of the last layer of the GRU model is taken as the total fixed-dimensional image encoding vector.

[0066] Specifically, for the agent environment image data at each moment, Inception, which has excellent performance in image processing tasks, is used as a feature extractor to mine image frame-level features.

[0067] Original image sequence data (x t-K , x t-K+1 , ..., x t , ..., x t+K-1 , x t+K ) is input into the Inception network model to obtain the image frame-level sequence feature representation output by the model (i t-K ,i t-K+1 , ..., i t , ..., i t+K-1 ,i t+K ), the judgment of key behavior actions needs to be based on the environment before the intelligent agent's behavior action and the changes in environmental information after the behavior action. Therefore, the above image frame level sequence feature representation (i t-K ,i t-K+1 , ..., i t , ..., i t+K-1 ,i t+K ), then use the GRU network to model the dependencies between sequences, input the GRU model and take the output of the last layer of the GRU model as the total fixed-dimensional image encoding vector imgIn t .

[0068] In one embodiment, the attention mechanism is calculated using the following calculation formula:

[0069]

[0070] Where the query vector is defined as Q = x α W Q , the key vector is K = x β W K , the value vector is V = x β W V , where W Q , W K , W V is the weight matrix, d k is the dimension of the value vector K, x α and x β are feature vectors of different modes, which are image feature encoding vectors imgIn in the present invention. t And the text feature encoding vector textIn t .

[0071] Specifically, the extracted image encoding vector imgIn t With the text encoding vector textInt , using the attention mechanism to achieve multimodal feature fusion, as shown in the attached Figure 2 The network diagram of multimodal feature fusion using the attention mechanism is shown in FIG. In the present invention, the query vector of the attention mechanism is defined as Q = x α W Q , the key vector is K = x β W K , the value vector is V = x β W V , where W Q , W K , W V is the weight matrix, x α and x β are feature vectors of different modes, which are image feature encoding vectors imgIn in the present invention. t And the text feature encoding vector textIn t ,Through the multimodal attention mechanism, the features of the two input modalities can be fused.

[0072] In one embodiment, the binary cross entropy function:

[0073]

[0074] Define a gradient modulus as:

[0075]

[0076] The gradient modulus g represents the distance between the true value and the predicted value of the sample.

[0077] In one embodiment, the gradient density function is defined as the number of samples distributed within a unit value region in the following form:

[0078]

[0079] where g k represents the gradient of the kth sample, and

[0080]

[0081] So the gradient density function GD(g) means that the gradient falls in the region The number of samples.

[0082] In one embodiment, the GHM Loss for classification is defined as follows:

[0083]

[0084] After the entire neural network model is built and the loss function is selected:

[0085] Use {(img 1 ,text 1 ;label 1 ),...,{(img t ,text t ;label t ),...,{(img T ,text T ;label T )} dataset for model training.

[0086] In addition, in the present invention, the proportion of key behavioral actions may be very low, and the classification problem is a category imbalance problem. Therefore, GHM Loss is used as the classification loss function in the present invention. The basic idea of ​​GHM Loss is to focus on those hard examples that are difficult to predict. At the same time, some hard examples are likely to be outliers, and these outliers should not be paid attention to by the model. GHM is a reconciliation mechanism that allows the model to continue to pay attention to hard examples while ignoring these outlier samples.

[0087] Beneficial effects:

[0088] (1) By fusing the features of multimodal data, we can effectively complement each other’s features and improve the recognition accuracy of key behaviors;

[0089] (2) The attention mechanism is used to achieve the fusion of multimodal features, focusing more on the modal information that contributes more to classification, further improving the recognition accuracy of action detection;

[0090] (3) The GHM Loss function is used to improve the recognition effect under class imbalance conditions.

[0091] The specific steps of a method for detecting key behaviors of intelligent agents based on feature fusion are as follows:

[0092] S1. Obtain multimodal training sample set

[0093] When the agent is simulating or running, it collects the environmental image data between the K time steps before and after the action time t of the agent to form the agent environmental image sequence data (x t-K , x t-K+1 , ..., x t , ..., x t+K-1 , x t+K ), denoted as img t , collect the current action information as the action text dataset, denoted as text t, and then human experts manually review and annotate the behavior to generate a category label for whether it is a key behavior. t , forming a pair of agent environment image data img t and the agent's behavior text data t And the category label label t , as the training data for the key behavior detection model (img t ,text t ;label t ). Repeat the above steps at different times to form the final model training sequence data set {(img 1 ,text 1 ;label 1 ),...,{(img t ,text t ;label t ),...,{(img T ,text T ;label T )};.

[0094] S2. Use the image encoding module to encode the features of the agent environment image data

[0095] s21. For the agent environment image data at each moment, the Inception model with excellent performance in image processing tasks is used as a feature extractor to mine the image frame-level features. Specifically, the Inception-v4 network is used to extract the frame-level agent environment RGB image sequence data (x t-K , x t-K+1 , ..., x t , ..., x t+K-1 , x t+K ) is input into the Inception-v4 network model to obtain the image frame-level sequence feature representation output by the model (i t-K ,i t-K+1 , ..., i t , ..., i t+K-1 ,i t+K );

[0096] S22. The judgment of key actions needs to be based on the environment before the agent's action and the changes in the environment after the action. Therefore, the image frame-level sequence feature representation (i t-K ,i t-K+1 , ..., i t , ..., i t+K-1 ,it+K ), use the GRU network to model the dependencies between sequences, input the GRU model and take the output of the last layer of the GRU model as the total fixed-dimensional image encoding vector imgIn t , complete the feature encoding of the agent environment image sequence;

[0097] S3. Use the text encoding module to encode the features of the agent's action text data

[0098] Each agent's action text data t Use the pre-trained ALBERT model to encode to obtain rich and diverse text representations and obtain a fixed-dimensional action text encoding vector textIn t ALBERT is a pre-trained language model that further enhances the generalization ability of the model and can better represent semantic information in different contexts.

[0099] S4. Classify the multimodal feature vector fused with the image encoding vector and the text encoding vector using an attention-based multimodal neural network

[0100] The extracted image encoding vector imgIn t With the text encoding vector textIn t , using the attention mechanism to achieve multimodal feature fusion, as shown in the attached Figure 2 The network diagram of multimodal feature fusion using the attention mechanism is shown in FIG. In the present invention, the query vector of the attention mechanism is defined as Q = x α W Q , the key vector is K = x β W K , the value vector is V = x β W V , where W Q , W K , W V is the weight matrix, x α and x β are feature vectors of different modes, which are image feature encoding vectors imgIn in the present invention. t And the text feature encoding vector textIn t , through the multimodal attention mechanism, the two input modal features can be fused, and the specific calculation formula is as follows:

[0101]

[0102] Through the above attention calculation, the attention feature vector representation after multimodal fusion is obtained, and then the category output is performed through the fully connected layer, so as to realize the detection of key behavioral actions.

[0103] In addition, in the present invention, the proportion of key behavioral actions may be very low, and the classification problem is a category imbalance problem. Therefore, GHM Loss is used as the classification loss function in the present invention. The basic idea of ​​GHM Loss is to focus on those hard examples that are difficult to predict. At the same time, some hard examples are likely to be outliers, and these outliers should not be paid attention to by the model. GHM is a reconciliation mechanism that allows the model to continue to pay attention to hard examples while ignoring these outlier samples.

[0104] Consider the binary cross entropy function of the present invention:

[0105]

[0106] Define a gradient modulus as:

[0107]

[0108] The gradient modulus g represents the distance between the true value and the predicted value of the sample.

[0109] The gradient density function is defined as the number of samples distributed within the unit value area in the following form:

[0110]

[0111] where g k represents the gradient of the kth sample, and

[0112]

[0113] So the gradient density function GD(g) means that the gradient falls in the region The number of samples.

[0114] Therefore, the GHM Loss for classification is defined as follows:

[0115]

[0116] After the entire neural network model is built and the loss function is selected, use {(img 1 ,text 1 ;label 1 ),...,{(img t ,text t ;label t ),...,{(img T ,text T ;label T )} dataset for model training.

[0117] The above embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention. The protection scope of the present invention is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present invention, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.

Claims

1. A method for detecting key behaviors of intelligent agents based on feature fusion, characterized in that: The method for detecting key behaviors of an intelligent agent based on feature fusion includes: Acquire a multimodal training sample set, wherein the training sample set includes sequence data consisting of paired environment image data of an intelligent agent and labeled behavior action text data of the intelligent agent; The environment image data of the intelligent agent is subjected to feature extraction using an image encoding module based on Inception and GRU neural networks to generate an image encoding vector of fixed dimension through key judgment; The text data of the agent's behavior t Use the ALBERT-based text encoding module to encode text features and obtain a fixed-dimensional action text encoding vector. The image coding vector and the action text coding vector are fused and then output as categories through a fully connected layer, thereby realizing the detection of key behavioral actions.

2. The method for detecting key behaviors of an intelligent agent based on feature fusion according to claim 1, characterized in that: The multimodal training sample set is obtained, wherein the training sample set includes sequence data consisting of paired environment image data of intelligent agents and labeled behavior action text data of intelligent agents, including: When the agent is simulating or running, the environment image data between the K time steps before and after the action time t of the agent is collected to form the environment image sequence data of the agent; Collect the current action information as the action text dataset; Receive training data for generating a detection model by marking the action text data set, wherein the training data for the detection model includes a pair of environment image data of an intelligent agent and behavior action text data of the intelligent agent, as well as a category label; Repeat the above process to finally generate a model training sequence dataset.

3. The method for detecting key behaviors of an intelligent agent based on feature fusion according to claim 2, characterized in that: The process of using an image encoding module based on Inception and GRU neural network to extract features of the environment image data of the agent through key judgment to generate an image encoding vector of fixed dimension comprises: For the agent environment image data at each moment, Inception, which has excellent performance in image processing tasks, is used as a feature extractor to mine image frame-level features and output image frame-level sequence features; The GRU network is used to model the dependencies between sequences. After inputting the GRU model, the output of the last layer of the GRU model is taken as the total fixed-dimensional image encoding vector.

4. The method for detecting key behaviors of an intelligent agent based on feature fusion according to claim 3, characterized in that: The attention mechanism is calculated using the following formula: The query vector is defined as Q = x α W Q , the key vector is K = x β W K , the value vector is V = x β W V , where W Q , W K , W V is the weight matrix, d k is the dimension of the value vector K, x α and x β are feature vectors of different modes, which are image feature encoding vectors imgIn in the present invention. t And the text feature encoding vector textIn t .

5. The method for detecting key behaviors of an intelligent agent based on feature fusion according to claim 4, characterized in that: Binary cross entropy function: Define a gradient modulus as: The gradient modulus g represents the distance between the true value and the predicted value of the sample.

6. The method for detecting key behaviors of an intelligent agent based on feature fusion according to claim 5, characterized in that: The gradient density function is defined as the number of samples distributed within the unit value area in the following form: where g k represents the gradient of the kth sample, and So the gradient density function GD(g) means that the gradient falls in the region The number of samples.

7. The method for detecting key behaviors of an intelligent agent based on feature fusion according to claim 6, characterized in that: The GHM Loss for classification is defined as follows: After the entire neural network model is built and the loss function is selected: Use {(img1, text1; label1), ..., {(img t ,text t ;label t ),...,{(img T ,text T ;label T )} dataset for model training.

Images