An interpretable target group behavior discrimination method

Abstract

The application belongs to the technical field of situation awareness, and specifically discloses an interpretable target group behavior discrimination method, which comprises the following steps: acquiring multi-source situation information data of a detected target group; obtaining a space-time distribution feature vector of the detected target based on the multi-source situation information data; constructing a to-be-learned double-layer graph structure based on the multi-source situation information data and the space-time distribution feature vector; discriminating the behavior category of the detected target group through a relational graph neural network based on the space-time distribution feature vector and the to-be-learned graph structure; and making the discrimination result of the relational graph neural network interpretable through a deep structure causal model SCM.

Description

Technical Field

[0001] This application belongs to the field of situational awareness technology, and more specifically, relates to an interpretable method for discriminating the behavior of a target group. Background Technology

[0002] In situational awareness, focusing solely on individual behavioral analysis wastes operational resources and makes one vulnerable to being misled by the opponent's tactical actions. Considering that individual behavior is often guided by the group, comprehensive behavioral analysis of the target group is often more important than single-target analysis. Understanding group behavior is the prerequisite and foundation for selecting core targets and implementing focused defense.

[0003] As detection methods become increasingly diversified, the identification of target group behavior has expanded from relying solely on track data to include multiple modalities such as textual descriptions, optical images, and even video. The introduction of multimodal data facilitates more accurate identification of target group behavior. However, due to the significant differences in properties among heterogeneous multimodal data sources, deep learning analysis methods must be introduced to effectively utilize them. However, widely used end-to-end deep learning methods require substantial training data, and their effectiveness heavily depends on data quality. In complex environments, missing information and incomplete target information acquisition are common, making it difficult to meet the training requirements of the model in terms of both quantity and quality. Furthermore, while these deep learning methods can improve the accuracy of group behavior analysis, their data-driven model training and "black box" internal structure make them uninterpretable. For combat missions, this lack of interpretability implies potential decision-making risks, representing a serious deficiency. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this application is to provide an interpretable target group behavior discrimination method, aiming to solve the problems of uninterpretable and unreliable behavior discrimination results of existing multimodal methods.

[0005] To achieve the above objectives, in a first aspect, this application provides an interpretable method for determining target group behavior, comprising:

[0006] Acquire multi-source situational information data of the target group;

[0007] Based on the multi-source situational information data, the spatiotemporal distribution feature vector of the detected target is obtained;

[0008] Based on the multi-source situational information data and the spatiotemporal distribution feature vector, a two-layer graph structure to be learned is constructed.

[0009] Based on the spatiotemporal distribution feature vector and the graph structure to be learned, the behavior category of the detected target group is determined by a relational graph neural network.

[0010] The discrimination results of the relational graph neural network are interpretable by using a deep structural causal model (SCM).

[0011] This application acquires multi-source situational information data of a target group across modalities, extracts spatiotemporal distribution feature vectors from the data as input to a relational graph neural network, and combines a two-layer graph structure for group behavior prediction learning and structural causal learning. This enables the relational graph neural network to learn the features of the spatiotemporal data in the overall structure and provide the target group behavior category judgment result. Then, a deep structural causal model is used to obtain an interpretable causal relationship for the judgment result, making the behavior judgment result interpretable and improving reliability.

[0012] According to the interpretable target group behavior discrimination method provided by the present invention, the step of acquiring multi-source situational information data of the detected target group includes:

[0013] The system acquires multi-source situational information data of the target group and stores and manages the multi-source situational information data in the form of dual-index "key-value" pairs.

[0014] This application uses a dual-index "key-value" pair format to store and manage data, which can effectively organize and manage large-scale and complex target detection data, and achieve fast data storage and efficient use.

[0015] According to the interpretable target group behavior discrimination method provided by the present invention, obtaining the spatiotemporal distribution feature vector of the detected target based on the multi-source situational information includes:

[0016] A pre-trained deep learning multimodal encoder is used to encode multi-source situational information data of different modalities using different initialization methods to obtain initial vectors;

[0017] The time and location information of the multi-source situational information data are concatenated to the end of the initial vector. The concatenated vector is used as the input of a pre-trained multimodal Chinese open-source large model. A Linear Layer is concatenated at the output position of the large model to adjust the data distribution and obtain the spatiotemporal distribution feature vector of the large model output.

[0018] This application employs a pre-trained deep learning multimodal encoder, which encodes multi-source situational information data of different modalities using different initialization methods, and concatenates time and location information to the end of the initial vector. The concatenated vector is used as the input of a pre-trained multimodal Chinese open-source large model, and a Linear Layer is concatenated at the output position of the large model to adjust the data distribution, which can fully reflect the spatiotemporal characteristics of the data.

[0019] According to the present invention, an interpretable target group behavior discrimination method is provided, wherein the step of constructing a two-layer graph structure to be learned based on the multi-source situational information data and the spatiotemporal distribution feature vector includes:

[0020] Based on the multi-source situational information data, the lower instance layer in the learning graph structure is constructed;

[0021] Based on the spatiotemporal distribution feature vector and the lower instance layer, the upper attribute layer in the learning graph structure is constructed.

[0022] This application uses all situation-related data collected, including environment, location, track, and images, as instances. Based on their interrelationships, these are incorporated into a graph structure. Then, based on the specific instances in the lower layer, attribute nodes in the upper layer are abstracted to construct an upper attribute layer graph structure. This facilitates the relational graph neural network to better predict the behavior categories of the detected target group.

[0023] According to the present invention, an interpretable target group behavior discrimination method is provided, wherein the behavior category of the detected target group is determined by a relational graph neural network based on the spatiotemporal distribution feature vector and the learning graph structure, including:

[0024] Based on the spatiotemporal distribution feature vector and the graph structure to be learned, the nodes in the relational graph neural network are encoded to obtain multiple group objectives.

[0025] While maintaining the learning complexity of the relational graph neural network, the vector addition calculation is directly adopted for multiple group targets to gather the features of the same group of targets together and obtain the feature set vector;

[0026] The feature set vector is passed through a multi-layer neural network to determine the behavioral category of the detected target group.

[0027] According to the present invention, an interpretable target group behavior discrimination method is provided, wherein the discrimination result of the relational graph neural network is made interpretable through a deep structural causal model (SCM), comprising:

[0028] By using a deep structural causal model (SCM) and employing counterfactual logic, the causal relationships between nodes in the attribute layer are learned based on the mapping from the instance layer to the attribute layer, making the discrimination results of the relational graph neural network interpretable.

[0029] This application uses a deep structural causal model and counterfactual logic to learn the causal relationships between nodes in the attribute layer based on the mapping from the instance layer to the attribute layer, thereby gaining a deep and comprehensive understanding of the target behavior type and its causes.

[0030] According to the interpretable target group behavior discrimination method provided by the present invention, after the discrimination result of the relational graph neural network is made interpretable through the deep structural causal model (SCM), the method further includes:

[0031] Based on the causal relationship, complete the lower instance layer in the two-layer graph structure to be learned;

[0032] The relational graph neural network is fine-tuned based on the completed lower instance layer.

[0033] This application mitigates the decline in target group behavior discrimination accuracy caused by missing and incomplete information by using the learned causal relationships to guide the fine-tuning of the relational graph neural network.

[0034] Secondly, this application provides an interpretable target group behavior discrimination device, comprising:

[0035] The first acquisition module is used to acquire multi-source situational information data of the detected target group;

[0036] The second acquisition module is used to obtain the spatiotemporal distribution feature vector of the detected target based on the multi-source situational information data;

[0037] The construction module is used to construct a two-layer graph structure to be learned based on the multi-source situational information data and the spatiotemporal distribution feature vector;

[0038] The discrimination module is used to determine the behavior category of the detected target group based on the spatiotemporal distribution feature vector and the graph structure to be learned, using a relational graph neural network.

[0039] An interpretable module is used to make the discrimination results of the relational graph neural network interpretable through a deep structural causal model (SCM).

[0040] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the interpretable target group behavior discrimination method described in the first aspect or any possible implementation thereof.

[0041] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the interpretable target group behavior discrimination method described in the first aspect or any possible implementation of the first aspect.

[0042] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to execute the interpretable target group behavior discrimination method described in the first aspect or any possible implementation of the first aspect.

[0043] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0044] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art:

[0045] The system acquires multi-source situational information data of the target group across modalities, extracts spatiotemporal distribution feature vectors from the data as input to the relational graph neural network, and combines a two-layer graph structure for group behavior prediction learning and structural causal learning. This enables the relational graph neural network to learn the features of the spatiotemporal data in the overall structure and provide the target group behavior category judgment result. Then, a deep structural causal model is used to obtain interpretable causal relationships for the judgment result, making the behavior judgment result interpretable and improving reliability. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0047] Figure 1 This is one of the flowcharts illustrating an interpretable target group behavior discrimination method provided in the embodiments of this application;

[0048] Figure 2 This is the second flowchart illustrating the interpretable target group behavior discrimination method provided in the embodiments of this application;

[0049] Figure 3 This is a schematic diagram of the structure of an interpretable target group behavior discrimination device provided in an embodiment of this application;

[0050] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0053] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0054] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0055] Next, combined Figures 1-2 This application introduces an interpretable target group behavior discrimination method provided in its embodiments.

[0056] Figure 1 This is one of the flowcharts illustrating an interpretable target group behavior discrimination method provided in this application embodiment, such as... Figure 1 As shown, the method includes the following steps:

[0057] Step 100: Acquire multi-source situational information data of the target group;

[0058] Optionally, the multi-source situational information data can be situational information obtained by fusing the time and location information of the target group, the multimodal information of the target group, etc.

[0059] Optionally, this application does not limit the acquisition and fusion methods of multi-source situational information data.

[0060] Step 110: Based on multi-source situational information data, obtain the spatiotemporal distribution feature vector of the detected target;

[0061] After obtaining multi-source situational information data, a distribution feature vector of spatiotemporal trajectory data is constructed. This spatiotemporal data is transformed into data that a relational graph neural network can understand in order to facilitate subsequent category judgment and classification. This data is used as the input variable of the relational graph neural network. Distribution features are extracted from the constructed temporal feature vector as the classification criterion of the deep learning network to capture the distribution features of the complete trajectory data.

[0062] Step 120: Construct a two-layer graph structure to be learned based on multi-source situational information data and spatiotemporal distribution feature vectors;

[0063] This application constructs a two-layer graph structure to be learned based on multi-source situational information data and spatiotemporal distribution feature vectors. The graph structure includes an interrelated lower instance graph and an upper attribute layer.

[0064] Step 130: Based on the spatiotemporal distribution feature vector and the graph structure to be learned, the behavior category of the target group is determined by the relational graph neural network;

[0065] For graph structures containing multiple types of relationships, Relational Graph Convolutional Networks (RGCNs), which can distinguish multiple relationships in the field of graph representation learning, are well-suited for learning complex graph relationships.

[0066] The core of RGCN lies in the encoding of nodes. This application encodes the nodes of RGCN based on spatiotemporal distribution feature vectors and the structure of the graph to be learned, thereby determining the behavior category of the target group being detected.

[0067] Step 140: The discrimination results of the relational graph neural network are made interpretable through the deep structural causal model (SCM).

[0068] In the field of interpretable learning, structural causal models (SCM) are widely used. SCM is a generalization and improvement of Bayesian networks, possessing the advantages of traditional Bayesian networks, and adding the structural causal model (SEM) commonly used in engineering. This allows SCM to remove the influence of a large number of unknown variable values ​​by using marginalization integration operations, thus enabling smooth probability-based causal inference.

[0069] In the field of image recognition, some studies have built the DeepSCM model based on the principle of SCM and using deep learning methods. DeepSCM can not only accurately identify image categories, but also realize the mining of causal relationships that affect the image presentation effect, which provides a reference for causal reasoning research of deep models in other fields.

[0070] This application applies deep SCM to perform causal reasoning learning at the attribute layer, thereby achieving the purpose of counterfactual logic judgment in target task interpretation, deeply and comprehensively understanding the target behavior type and its causes, and making the discrimination results of the relational graph neural network interpretable.

[0071] This application provides an interpretable target group behavior discrimination method. First, it acquires multi-source situational information data of the target group across modalities, extracts spatiotemporal distribution feature vectors from the data as input to a relational graph neural network, and combines a two-layer graph structure for group behavior prediction learning and structural causal learning. This enables the relational graph neural network to learn the features of the spatiotemporal data in the overall structure and provide the target group behavior category judgment result. Then, a deep structural causal model is used to obtain an interpretable causal relationship for the discrimination result, making the behavior discrimination result interpretable and improving reliability.

[0072] In some embodiments, step 100 specifically includes:

[0073] Step 1001: Obtain multi-source situational information data of the target group and store and manage the multi-source situational information data in the form of dual-index "Key-Value" pairs.

[0074] Because situational information possesses typical temporal and spatial attributes, and the detection cycle of mainstream detection equipment is generally one second, the scale of detection data becomes extremely large as the time frame extends further during detection missions. Moreover, situational information primarily includes the target's temporal and location information, and depending on the detection method, it often also includes target text reports, target detection images, and other information. Target detection data contains multiple modalities, resulting in a complex data composition. Furthermore, various correlations exist among these data points, further exacerbating the data complexity.

[0075] In order to effectively organize and manage large-scale and complex target detection data, this application adopts a dual-index "key-value" key-value pair format to store and manage the data, thereby achieving fast data storage and efficient utilization.

[0076] Specifically, a high-dimensional hash function is first used to hash all the data, obtaining a unique hash code for each piece of data. Then, a first-level index structure is constructed, using "[Time]-[Location]-[Data Hash Value]" as the underlying index key and "[Data]" as the value, to store all the data. The time and location are stored in the form of "year, month, day, hour, minute, second" and "precision latitude, altitude," respectively, with time accurate to the second and location accurate to six decimal places (i.e., accurate to the meter).

[0077] Next, a second-level index is constructed based on the types of relationships. First, the relationships between categorized data are identified. This is done using a rule-based approach, developed through thorough communication with experts who have worked on models based on preceding data relationships, to establish rules that differentiate between different types of relationships.

[0078] For example, there are common sequential relationships among target waypoint data, correlations between target waypoints and target images, correlations between target waypoints and target reports, and interrelationships between target waypoints and other target waypoints. Like data, relationships possess temporal and spatial attributes. However, unlike data, their spatial attributes involve two objects. Therefore, for the index storage of the relationship C between "data A -> data B", the index key is "[time] - [relationship type] - [hash value of data A]", and the value is "[hash value of data B - relationship C]", thus storing all data.

[0079] By constructing a dual index, efficient querying of data and its relationships at any time and location can be achieved. Furthermore, adding data and relationships can be done directly through a query. Deleting data can also be accomplished by querying based on the index key value.

[0080] In some embodiments, step 110 specifically includes:

[0081] Step 1101: A pre-trained deep learning multimodal encoder is used to encode multi-source situational information data of different modalities using different initialization methods to obtain initial vectors;

[0082] Step 1102: The time and location information of the multi-source situational information data are concatenated to the end of the initial vector. The concatenated vector is used as the input of the pre-trained multimodal Chinese open-source large model. A Linear Layer is concatenated at the output position of the large model to adjust the data distribution and obtain the spatiotemporal distribution feature vector of the large model output.

[0083] This application employs an open-source pre-trained deep learning multimodal encoder to construct a distribution feature vector of data with spatiotemporal characteristics using collected multi-source situational information data.

[0084] Large models based on Transformer have the potential to process spatiotemporal feature data. In recent years, based on general open source large models, multimodal time series large models such as VideoLLM and GPT4STM have shown superior performance in fields such as video processing and financial data processing.

[0085] In one embodiment of this application, based on Chinese open-source large-scale models such as GLM and VisCPM, and drawing on the time-series processing architecture of GPT4STM, different initialization methods are used for data of different modalities. For time-series data such as flight tracks, Linear Embedding is first used for encoding to obtain initial vectors; for text-type data, BERT Embedding is first used for encoding to obtain initial vectors; and for image-type data, Transformer Embedding is used for encoding to obtain initial vectors.

[0086] Furthermore, to fully reflect the spatiotemporal characteristics of the data, the "[time][location]" information of the data needs to be directly concatenated to the end of the initial vector. The concatenated vector is then used as the input to a pre-trained multimodal Chinese open-source large-scale model, and a Linear Layer is concatenated at the output of the large-scale model to adjust the data distribution. Throughout the entire model training process, the parameters of the large-scale model remain constant, utilizing its general understanding of multiple modal data to encode the aforementioned multimodal data with various spatiotemporal attributes.

[0087] In some embodiments, step 120 specifically includes:

[0088] Step 1201: Based on multi-source situational information data, construct the lower instance layer in the graph structure to be learned;

[0089] Step 1202: Based on the spatiotemporal distribution feature vector and the lower instance layer, construct the upper attribute layer in the graph structure to be learned.

[0090] This application first constructs a lower-level instance layer graph structure, taking all collected multi-source situational information data, including environment, location, track, image, and other situational-related data, as instances and incorporating them into the graph structure based on their interrelationships. Then, based on the specific instances in the lower layer, the attribute nodes in the upper layer are abstracted to construct the upper-level attribute layer graph structure.

[0091] Specifically, effective identification of group behavior requires the support of basic data such as the relative positions of targets, the equipment carried by targets, and the historical activities of targets. Therefore, it is necessary to include all situation-related data such as environment, location, track, and image into the data source of the graph structure required for behavior identification. Based on multi-source situation information data, a retrieval method is used to connect all the relationships contained therein to construct the lower-level instance graph.

[0092] Based on specific examples from the lower layer, unsupervised clustering and rule induction are used together to abstract the attribute nodes of the upper layer, constructing an upper-layer attribute graph structure. Specifically, the initial vector obtained from the encoding (which has not been processed by the large model) is first clustered using unsupervised clustering, grouping data with similar features together. It should be noted that since time and space information are easily used as clustering criteria, but this type of information is unrelated to attribute values, the initial vector used here does not carry "[time][location]" information.

[0093] Next, similar to the construction of the attribute layer and the classification of relationships, we will rely on domain experts to guide unsupervised clustering and summarize based on existing examples.

[0094] In some embodiments, step 130 specifically includes:

[0095] Step 1301: Based on the spatiotemporal distribution feature vector and the graph structure to be learned, the nodes in the relational graph neural network are encoded to obtain multiple group objectives;

[0096] Step 1302: While maintaining the learning complexity of the relational graph neural network, the features of multiple group targets are directly calculated by vector addition to gather the features of the same group of targets together to obtain the feature set vector.

[0097] Step 1303: The feature set vector is passed through a multi-layer neural network to determine the behavior category of the target group.

[0098] This application is approved by g m The (·) function encodes nodes in a relational graph neural network. In the standard RGCN, g m (·) is usually defined as a simple linear transformation with normalization capability, that is, Among them, e i and e j These are graph nodes e. i and e j The graph input vector, i.e., the spatiotemporal distribution feature vector, For node e i For e j The Laplacian matrix is ​​given by , and W is the transformation weight matrix of the neural network.

[0099] When the graph structure is complex, the use of the neural network transformation weight matrix W and autoregression will significantly increase the complexity of graph representation learning, thus seriously affecting the performance of the graph neural network model. At the same time, the negative impact of model complexity becomes more severe when there is a large amount of node data.

[0100] The spatiotemporal graph neural network involved in this application has a very large number of nodes. Therefore, this application will use the standard gm (·) is usually simplified to It can significantly reduce the complexity of node representation learning while improving model performance.

[0101] Based on the above improvements, from the perspective of training the model in the form of matrix updates, the update method of RGCN adopted in this application is as follows:

[0102]

[0103] in, Represents graph node e i In graph convolution, the vectors of the l-th layer are learned. Represents node e under relation r i The neighbor index set, in which It is a normalized constant for different relationships, which can be learned or pre-defined. Through the above-mentioned RGCN learning that distinguishes various types of relationships, the representation vectors carrying relationship information of each node can be learned on the topology of the instance layer graph.

[0104] Specifically, during the initialization of graph network nodes, for nodes that can be encoded, such as images and tracks, their codes of equal length are used as the initial vectors for graph learning; for other instance layer nodes, they are initialized in a random manner.

[0105] In the standard data sample for group target behavior discrimination (Ground Truth Sample), a group of targets constitutes a group. This group is labeled with a certain behavior towards one or more targets outside the group. The behavior of the target group towards other targets can be patrol, tracking, surveillance, air defense, anti-submarine warfare, etc. However, due to the uncertainty of the number of targets within the group, pattern matching cannot be directly achieved using a multilayer neural network (MLP).

[0106] Considering that after learning through graph neural network representations, the target vectors all reside within the same vector space, allowing for simple vector computation, we directly employ vector addition for multiple target groups while maintaining the complexity of graph neural network learning. This gathers the features of the same group of targets together, obtaining a feature set vector. This feature set vector is then passed through a multilayer perceptron (MLP) to discriminate the behavior of the target group.

[0107] In some embodiments, step 140 specifically includes:

[0108] Step 1401: Using a deep structural causal model (SCM) and a counterfactual logic approach, the causal relationships between nodes in the attribute layer are learned based on the mapping from the instance layer to the attribute layer, making the discrimination results of the relational graph neural network interpretable.

[0109] Since graph convolutional networks (GCNNs) operate at the instance layer for group behavior recognition, they learn representations of the graph nodes in the instance layer through parameter learning within the network. Then, they apply the learned representations in conjunction with MLPs to predict the type of target group behavior. This process has the characteristics of typical deep learning methods, and its inference process is difficult for humans to understand.

[0110] This application uses deep SCM, a causal reasoning learning method that can learn the causal relationships between nodes in the attribute layer by using counterfactual logic judgment based on the relationship between the instance layer and the attribute layer. This allows for a deep and comprehensive understanding of the target behavior type and its causes, making the discrimination results of the relational graph neural network interpretable.

[0111] Deep SCM is composed of MLP combined with attribute layer graph structure. In terms of training, it uses reward mechanism and stochastic gradient descent technique to automatically learn the parameters in MLP.

[0112] Furthermore, this technique uses an MLP structure to model independent causal mechanisms. Each MLP's input layer contains M×N neurons, specifically M N-dimensional one-hot vectors. The intermediate hidden layers can be arbitrarily chosen, generally consisting of Max(4M, 4N) neurons. The subsequent activation layers use a LeakyReLU structure. The final output layer contains N neurons, representing the log probability of each task classification.

[0113] In some embodiments, after step 140, the method further includes:

[0114] Step 150: Complete the lower instance layer in the two-layer graph structure to be learned based on causal relationships;

[0115] Step 160: Fine-tune the relational graph neural network based on the completed lower instance layer.

[0116] Figure 2 This is a second flowchart illustrating the interpretable target group behavior discrimination method provided in the embodiments of this application, such as... Figure 2As shown, in one embodiment of this application, expert knowledge and learned causal relationships are stored in a tree structure. Then, a heuristic search method is used to match the causal relationships with existing expert knowledge, further improving the causal analysis chain. Finally, a feedback learning approach is adopted to attempt to infer the lower instance layer of the completed graph structure using the obtained causal relationship rules. Based on the trained RGCN model parameters, fine-tuning is performed using new instance layer data to achieve more accurate group behavior prediction.

[0117] The following is combined with Figure 3 An interpretable target group behavior discrimination device provided by the present invention is described below. The interpretable target group behavior discrimination device described below can be referred to in correspondence with the interpretable target group behavior discrimination method described above.

[0118] Figure 3 This is a schematic diagram of the structure of an interpretable target group behavior discrimination device provided in an embodiment of this application, such as... Figure 3 As shown, the device includes a first acquisition module 310, a second acquisition module 320, a construction module 330, a discrimination module 340, and an interpretable module 350, wherein:

[0119] The first acquisition module 310 is used to acquire multi-source situational information data of the detected target group;

[0120] The second acquisition module 320 is used to obtain the spatiotemporal distribution feature vector of the detected target based on multi-source situational information data;

[0121] Module 330 is used to construct a two-layer graph structure to be learned based on multi-source situational information data and spatiotemporal distribution feature vectors.

[0122] The discrimination module 340 is used to discriminate the behavior category of the target group based on the spatiotemporal distribution feature vector and the graph structure to be learned, through a relational graph neural network.

[0123] Interpretable module 350 is used to make the discrimination results of relational graph neural networks interpretable through deep structural causal model SCM.

[0124] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0125] Based on the methods in the above embodiments, Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4As shown in the illustration, this application provides an electronic device that may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440. The processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can invoke logical instructions stored in the memory 430 to execute the interpretable target group behavior discrimination method described in the above embodiment.

[0126] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the interpretable target group behavior discrimination method described in the various embodiments of this application.

[0127] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the interpretable target group behavior discrimination method in the above embodiments.

[0128] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the interpretable target group behavior discrimination method in the above embodiments.

[0129] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0130] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0131] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0132] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0133] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An interpretable target group behavior discrimination method, characterized in that, The method comprises the following steps: obtaining multi-source situation information data of a detection target group; obtaining a space-time distribution feature vector of the detection target based on the multi-source situation information data; constructing a to-be-learned double-layer graph structure based on the multi-source situation information data and the space-time distribution feature vector; determining a behavior category of the detection target group by a relational graph neural network based on the space-time distribution feature vector and the to-be-learned double-layer graph structure; making the determination result of the relational graph neural network interpretable by a deep structural causal model SCM; the step of constructing the to-be-learned double-layer graph structure based on the multi-source situation information data and the space-time distribution feature vector comprises the following steps: constructing a lower instance layer in the to-be-learned graph structure based on the multi-source situation information data; constructing an upper attribute layer in the to-be-learned graph structure based on the space-time distribution feature vector and the lower instance layer; the step of making the determination result of the relational graph neural network interpretable by the deep structural causal model SCM comprises the following step: learning a causal relationship between attribute layer nodes according to a relationship of the instance layer mapping to the attribute layer by the deep structural causal model SCM in a counterfactual logic determination manner, so as to make the determination result of the relational graph neural network interpretable. 2.The interpretable target group behavior discrimination method of claim 1, wherein, the step of obtaining the multi-source situation information data of the detection target group comprises the following steps: obtaining multi-source situation information data of a detection target group, and storing and managing the multi-source situation information data in a form of a double index "Key-Value" key-value pair. 3.The interpretable target group behavior discrimination method of claim 1, wherein, the step of obtaining the space-time distribution feature vector of the detection target based on the multi-source situation information comprises the following steps: using a pre-trained deep learning multi-modal encoder to encode multi-source situation information data of different modalities in different initialization manners to obtain an initial vector; splicing time information and location information of the multi-source situation information data to the end of the initial vector, taking the spliced vector as an input of a pre-trained multi-modal Chinese open source large model, and splicing a LinearLayer at an output position of the large model to adjust the distribution of data, to obtain a space-time distribution feature vector output by the large model. 4.The interpretable target group behavior discrimination method of claim 1, wherein, the step of determining the behavior category of the detection target group by the relational graph neural network based on the space-time distribution feature vector and the to-be-learned graph structure comprises the following steps: encoding nodes in the relational graph neural network based on the space-time distribution feature vector and the to-be-learned graph structure to obtain a plurality of group targets; under the premise of maintaining the complexity of learning of the relational graph neural network, directly adopting a vector addition calculation form to the plurality of group targets to collect features of the same group target together to obtain a feature set vector; judging the behavior category of the detection target group by the feature set vector through a multi-layer neural network. 5.The interpretable target group behavior discrimination method of claim 1, wherein, after the step of making the determination result of the relational graph neural network interpretable by the deep structural causal model SCM, the method further comprises the following steps: completing the lower instance layer in the to-be-learned double-layer graph structure based on the causal relationship; fine-tuning the relational graph neural network based on the completed lower instance layer. 6.An interpretable target group behavior discrimination apparatus characterized by, The method comprises the following steps: a first obtaining module is configured to obtain multi-source situation information data of a detection target group; a second acquisition module, configured to acquire a space-time distribution feature vector of the detection target based on the multi-source situation information data; a construction module, configured to construct a to-be-learned double-layer graph structure based on the multi-source situation information data and the space-time distribution feature vector; a discrimination module, configured to discriminate a behavior category of the detection target group based on the space-time distribution feature vector and the to-be-learned double-layer graph structure through a relational graph neural network; an interpretable module, configured to make a discrimination result of the relational graph neural network interpretable through a deep structural causal model SCM; the construction module is specifically configured to: construct a lower instance layer in the to-be-learned graph structure based on the multi-source situation information data; construct an upper attribute layer in the to-be-learned graph structure based on the space-time distribution feature vector and the lower instance layer; the interpretable module is specifically configured to: learn a causal relationship between attribute layer nodes according to a relationship of the instance layer mapping to the attribute layer in a way of counterfactual logic judgment through the deep structural causal model SCM, so as to make the discrimination result of the relational graph neural network interpretable.

7. An electronic device, comprising: comprise: at least one memory for storing a computer program; at least one processor for executing the program stored in the memory, and when the program stored in the memory is executed, the processor is configured to execute the interpretable target group behavior discrimination method according to any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: when the computer program runs on the processor, the processor is caused to execute the interpretable target group behavior discrimination method according to any one of claims 1-5.

Images