Bank security abnormal behavior recognition method and system based on multi-modal data

Abstract

The application discloses a bank security abnormal behavior recognition method and system based on multi-modal data, relates to the technical field of abnormal behavior recognition, and comprises the following steps: acquiring a plurality of heterogeneous sensing nodes of a bank security system, and respectively arranging multi-modal sensing modules; performing multi-source behavior sensing to obtain a multi-source behavior sensing network sequence; performing security risk topology separation; performing double security abnormal behavior recognition to obtain an abnormal behavior recognition result; performing adaptive scheduling to obtain a scheduling security disposal instruction, and performing unmanned vehicle or unmanned aerial vehicle cooperative verification. The application solves the technical problems that the existing technology has single bank security data sensing dimension, lacks topology analysis, and thus causes low abnormal behavior recognition accuracy and untimely response, achieves the fusion sensing of multi-modal data of a bank security whole process and the accurate separation of a risk topology, and improves the accuracy and timeliness of bank security abnormal behavior recognition.

Description

Technical Field

[0001] This invention relates to the field of abnormal behavior recognition technology, specifically to a method and system for recognizing abnormal behavior in bank security based on multimodal data. Background Technology

[0002] In the field of bank security, security scenarios cover internal office areas, business areas, and surrounding external areas. Comprehensive monitoring and timely handling of various security risks, such as abnormal personnel behavior, fire hazards, and unauthorized intrusion, are required. Existing bank security systems mostly use single-modal sensing devices to collect data, resulting in limited sensing dimensions and a lack of effective correlation analysis between heterogeneous sensing nodes, making it difficult to build a comprehensive security behavior sensing network. Furthermore, traditional security anomaly identification methods do not accurately separate security risk topologies. A unified identification model cannot account for both routine anomalies in stable topologies and potential anomalies in risky topologies, easily leading to low anomaly identification accuracy and high false positive / false negative rates. Moreover, the identification results lack efficient linkage with security patrol strategies, making it impossible to quickly dispatch unmanned vehicles, drones, and other equipment for specialized collaborative verification. This results in delayed response times for security anomalies, failing to meet the bank's needs for full-process, intelligent, and highly timely monitoring and handling.

[0003] Existing technologies for bank security data perception suffer from limited dimensions and lack of topology analysis, resulting in low accuracy in identifying abnormal behavior and untimely response. Summary of the Invention

[0004] This application provides a method and system for identifying abnormal behavior in bank security based on multimodal data, which addresses the technical problems in existing technologies where the perception dimension of bank security data is singular and topological analysis is lacking, resulting in low accuracy and untimely response in identifying abnormal behavior.

[0005] In view of the above problems, this application provides a method and system for identifying abnormal security behaviors in banks based on multimodal data.

[0006] The first aspect of this application provides a method for identifying abnormal security behavior in banks based on multimodal data, the method comprising: Multiple heterogeneous sensing nodes of a bank's security system are acquired, and multimodal sensing modules are deployed at each of these nodes. The system traverses these nodes within a preset window to perform multi-source behavior sensing, obtaining a multi-source behavior sensing network sequence. This multi-source behavior sensing network sequence is then traversed to perform security risk topology separation, obtaining a consistency relationship subgraph sequence and a risk relationship subgraph sequence. Based on these two sequences, dual security anomaly behavior identification is performed to obtain anomaly behavior identification results. Adaptive scheduling of real-time security patrol strategies is then performed based on the anomaly behavior identification results to obtain scheduling security handling instructions. Finally, based on these instructions, unmanned vehicles or drones are used for collaborative verification of the bank's security system.

[0007] A second aspect of this application provides a bank security anomaly behavior recognition system based on multimodal data, the system comprising: The system includes a heterogeneous sensing node acquisition module, used to acquire multiple heterogeneous sensing nodes of the bank's security system and deploy multimodal sensing modules at each of these nodes; a multi-source behavior sensing module, used to traverse the multiple heterogeneous sensing nodes within a preset window to perform multi-source behavior sensing and obtain a multi-source behavior sensing network sequence; a security risk topology separation module, used to traverse the multi-source behavior sensing network sequence to perform security risk topology separation and obtain a consistency relationship subgraph sequence and a risk relationship subgraph sequence; an abnormal behavior identification result acquisition module, used to perform dual security abnormal behavior identification based on the consistency relationship subgraph sequence and the risk relationship subgraph sequence to obtain abnormal behavior identification results; and a collaborative verification module, used to adaptively schedule real-time security patrol strategies based on the abnormal behavior identification results, obtain scheduling security handling instructions, and perform unmanned vehicle or drone collaborative verification of the bank's security system based on the scheduling security handling instructions.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages: This method acquires multiple heterogeneous sensing nodes from a bank's security system and deploys multimodal sensing modules at each node. Multi-source behavior sensing is performed to obtain a multi-source behavior sensing network sequence. The network sequence is traversed to perform security risk topology separation, obtaining a consistency relationship subgraph sequence and a risk relationship subgraph sequence. Dual security anomaly behavior identification is performed to obtain anomaly behavior identification results. Adaptive scheduling is implemented to obtain scheduling security handling instructions. Based on these instructions, unmanned vehicles or drones are used for collaborative verification of the bank's security system. This achieves the technical effect of realizing the fusion perception of multimodal data and accurate risk topology separation throughout the entire bank security process, improving the accuracy and timeliness of anomaly behavior identification. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A schematic diagram of the bank security abnormal behavior identification method based on multimodal data provided in this application embodiment; Figure 2 A schematic diagram of the structure of a bank security abnormal behavior recognition system based on multimodal data provided in this application embodiment.

[0011] Figure labeling: Heterogeneous sensing node acquisition module 10, multi-source behavior sensing module 20, security risk topology separation module 30, abnormal behavior identification result acquisition module 40, collaborative verification module 50. Detailed Implementation

[0012] This application provides a method and system for identifying abnormal behavior in bank security based on multimodal data, which addresses the technical problems in existing technologies where the perception dimension of bank security data is singular and topological analysis is lacking, resulting in low accuracy and untimely response in identifying abnormal behavior.

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0014] Example 1, as Figure 1 As shown, this application provides a method for identifying abnormal security behavior in banks based on multimodal data, the method comprising: Step S100: Obtain multiple heterogeneous sensing nodes of the bank security system, and deploy multimodal sensing modules at each of the multiple heterogeneous sensing nodes.

[0015] Specifically, the system first acquires multiple heterogeneous sensing nodes covering the entire security scenario of the bank's internal business areas, office areas, computer rooms, external parking lots, and surrounding patrol areas. These heterogeneous sensing nodes are adapted to the sensing needs of different security areas within the bank and have differentiated sensing deployment attributes. Simultaneously, multimodal sensing modules are deployed at each heterogeneous sensing node. Through the comprehensive deployment of multimodal sensing modules, a multi-dimensional, full-scenario sensing hardware foundation covering the internal and external aspects of the bank is built for subsequent multi-source behavior sensing and abnormal behavior recognition. This enables the multimodal expansion of the bank's security system's sensing capabilities and ensures the comprehensiveness and diversity of subsequent security data collection.

[0016] Step S200: Traverse the multiple heterogeneous sensing nodes within a preset window to perform multi-source behavior sensing and obtain a multi-source behavior sensing network sequence.

[0017] Specifically, the process begins by using multimodal sensing modules deployed at each heterogeneous sensing node to comprehensively collect and perceive the internal and external security-related behaviors of multiple heterogeneous sensing nodes within a preset time window, obtaining multiple heterogeneous sensing node behavior data sequences for each node. Then, based on these multiple heterogeneous sensing node behavior data sequences, correlation analysis is conducted between the multiple heterogeneous sensing nodes. Each heterogeneous sensing node is first used as a source node, and associated nodes are extracted from the behavior data sequences to obtain multiple sets of associated node pairs. These sets of associated node pairs are then used as indexes to perform semantic parsing of the relationship between the behavior data sequences, determining multiple sets of semantic parsing results. Subsequently, each associated node pair is used as the two endpoints of a relationship edge, and the relationship edge is identified using the semantic parsing results, thus constructing multiple sets of relationship edges. Finally, based on the multiple heterogeneous sensing nodes and the constructed multiple sets of relationship edges, a multi-source behavior sensing network sequence is built, achieving networked and temporally integrated multi-source behavior data for bank security.

[0018] Step S300: Traverse the multi-source behavior perception network sequence to perform security risk topology separation, and obtain a consistent relationship subgraph sequence and a risk relationship subgraph sequence.

[0019] Specifically, firstly, multiple heterogeneous sensing nodes are associated with subnetworks from the multi-source behavior sensing network sequence according to the K-order association mechanism, resulting in multiple heterogeneous sensing node multi-source behavior sensing subnetwork sequences. Then, the first heterogeneous sensing node multi-source behavior sensing subnetwork sequence is extracted from this subnetwork sequence, and its structural features are analyzed to obtain a subnetwork structural feature sequence. Adjacency matrix analysis is performed on this structural feature sequence. First, the fine-grained similarity of adjacent subnetwork structural features is calculated and normalized before being filled into an empty matrix, resulting in an adjacency matrix sequence. Then, multi-scale approximate identification is performed on the adjacency matrix sequence to obtain a multi-scale approximate similarity set, and the similarity of this set is calculated. The average value is used to obtain the stability coefficient of the first heterogeneous sensing node and added to the set of stability coefficients of multiple heterogeneous sensing nodes. The stability coefficients of all heterogeneous sensing nodes are calculated sequentially. Then, the stability coefficients of each heterogeneous sensing node are compared with the preset stability coefficient threshold. If the coefficient is greater than or equal to the threshold, the corresponding node is marked as a consistent node. If it is less than the threshold, it is marked as a risk node. Finally, the heterogeneous sensing nodes with consistent node labels and risk node labels are extracted from the multi-source behavior sensing network sequence, and consistent relationship subgraph sequence and risk relationship subgraph sequence are constructed respectively to complete the security risk topology separation of the multi-source behavior sensing network sequence.

[0020] Step S400: Perform dual security abnormal behavior identification based on the consistency relationship subgraph sequence and the risk relationship subgraph sequence to obtain the abnormal behavior identification result.

[0021] Specifically, a dual security anomaly behavior identification operation is performed on the consistency relationship subgraph sequence and the risk relationship subgraph sequence obtained from the security risk topology separation. First, a regular anomaly identification network is invoked to identify security anomalies in the consistency relationship subgraph sequence, uncovering hidden regular security anomaly behavior features and outputting the corresponding first anomaly behavior identification result. Simultaneously, a risk anomaly identification network is invoked to identify specific security anomalies in the risk relationship subgraph sequence, accurately capturing the high-risk security anomaly behavior features represented in the sequence and obtaining the corresponding second anomaly behavior identification result. Finally, the first and second anomaly behavior identification results are comprehensively summarized and integrated to form a complete security anomaly behavior identification result covering regular anomalies and high-risk anomalies, providing accurate anomaly judgment basis for subsequent security inspection strategy scheduling and collaborative verification.

[0022] Step S500: Adaptively schedule the real-time security patrol strategy based on the abnormal behavior identification results, obtain the scheduling security handling instructions, and conduct unmanned vehicle or drone collaborative verification of the bank security system based on the scheduling security handling instructions.

[0023] Specifically, based on the abnormal behavior identification results obtained from dual security anomaly identification, the real-time security patrol strategy of the bank's security system is dynamically and adaptively scheduled. Combining key information such as the type of abnormal behavior, the area where it occurs, and the risk level, the patrol routes, patrol frequency, and patrol subjects are adjusted and optimized in a targeted manner, generating dispatch security handling instructions adapted to the current security anomaly situation. Based on these dispatch security handling instructions, the bank's unmanned vehicle and drone security execution units are coordinated and scheduled. According to the environmental characteristics and verification needs of the abnormal area, special verification tasks are assigned to unmanned vehicles and drones, realizing coordinated linkage verification of the bank's indoor and outdoor ground areas by unmanned vehicles and of high-altitude areas and complex terrain areas by drones. This completes the on-site verification, information feedback, and real-time handling of abnormal behavior, realizing closed-loop management of bank security anomaly behavior identification and handling, covering the anomaly verification needs of the entire security scenario, including the bank's internal business area, office area, computer room, and external parking lot and surrounding areas.

[0024] In one possible implementation, step S200 further includes: Step S210: The behavior of the multiple heterogeneous sensing nodes within a preset window is perceived by the multimodal sensing module to obtain a sequence of behavior data of the multiple heterogeneous sensing nodes.

[0025] Step S220: Based on the behavioral data sequence of the multiple heterogeneous sensing nodes, perform correlation analysis between the multiple heterogeneous sensing nodes to determine multiple sets of correlation edge sequences.

[0026] Step S230: Construct a multi-source sensing network sequence based on the multiple heterogeneous sensing nodes and multiple sets of associated edge sets.

[0027] Specifically, relying on the multimodal sensing modules pre-deployed at various heterogeneous sensing nodes in the bank's security system, synchronous and continuous behavioral sensing data is collected from multiple heterogeneous sensing nodes in the bank's internal business area, office area, computer room, external parking lot, and surrounding patrol area, according to a set preset time window. The multimodal sensing modules comprehensively capture and record various security-related behavioral data such as personnel activities, equipment operation, and environmental status within the coverage area of ​​each node, generating a corresponding behavioral data time sequence for each heterogeneous sensing node, and finally integrating them to form multiple heterogeneous sensing node behavioral data sequences, completely preserving the multi-dimensional security sensing data of each node within the preset window.

[0028] Based on the behavioral data sequences of multiple heterogeneous sensing nodes, this study conducts correlation analysis between multiple nodes. First, each heterogeneous sensing node is treated as a source node. Heterogeneous sensing nodes with data associations with each source node are extracted from the behavioral data sequences as associated nodes, forming multiple sets of associated node pairs. Then, using these sets of associated node pairs as indexes, semantic parsing of the correlation relationships between the multiple heterogeneous sensing node behavioral data sequences is performed to uncover the inherent logic and attribute features of the associations between nodes, determining multiple sets of semantic parsing results. Finally, each associated node pair in these sets is used as one of the two endpoints of an association edge, and the semantic parsing results are used to assign attributes and semantic labels to each association edge, completing the construction of multiple sets of association edge sequences and clearly defining the correlation relationships and characteristic attributes between heterogeneous sensing nodes.

[0029] Multiple heterogeneous sensing nodes in the bank security system are used as the core nodes of the network topology. A sequence of multiple relational edges obtained through correlation analysis is used as the relational connection edges between nodes. Based on the temporal characteristics within a preset window and the relational logic between nodes, each heterogeneous sensing node and its corresponding relational edge are structurally mapped and topologically constructed. The attribute information of the nodes, the semantic identification information of the edges, and the temporal dimension information are integrated into the network structure. The corresponding sensing network structure is generated sequentially according to the time sequence. Finally, a multi-source sensing network sequence is integrated and constructed to realize the fusion of multi-source sensing data of bank security from discrete node data to structured, temporal, and networked data.

[0030] In one possible implementation, step S220 further includes: Step S221: Using multiple heterogeneous sensing nodes as source nodes, extract the associated nodes in the behavioral data sequence of the multiple heterogeneous sensing nodes to obtain a set sequence of multiple associated node pairs.

[0031] Step S222: Using the sequence of multiple associated node pairs as an index, perform semantic parsing of the multiple heterogeneous sensing node behavior data sequences to determine multiple sets of semantic parsing results.

[0032] Step S223: Take each of the multiple associated node pairs in the set sequence as the two endpoints of the association relationship edge, and use the set sequence of association relationship semantic parsing results to identify the association relationship edge, and construct the multiple association relationship edge set sequence.

[0033] Specifically, for multiple heterogeneous sensing nodes in a bank security system, node association mining is carried out by treating each node as a source node. Using the behavioral data sequence corresponding to each source node as the core reference, heterogeneous sensing nodes with data association and behavioral linkage characteristics with the source nodes are accurately extracted from the full set of behavioral data sequences of multiple heterogeneous sensing nodes as associated nodes. Each source node is matched with a corresponding associated node to form a node pair. All node pairs are integrated according to time sequence and node attributes to finally obtain a set of multiple associated node pairs, clearly defining the basic association objects between each heterogeneous sensing node.

[0034] Using the acquired sequence of multiple associated node pairs as the core retrieval index, semantic parsing of association relationships is performed on the behavioral data sequences of multiple heterogeneous sensing nodes in the bank security system. Based on the temporal behavioral data corresponding to each associated node pair pointed to by the index, the inherent logic of the association between nodes, the type of behavioral linkage, the data association characteristics, and the semantic connotation in the security scenario are deeply mined. The key information such as the attributes, triggering conditions, and association strength of the association relationship between each node pair is accurately parsed. The parsing results of each set of associated node pairs are classified, integrated, and temporally aggregated, and finally, multiple sets of semantic parsing results matching the sequence of associated node pairs are determined.

[0035] Based on the obtained sequence of multiple associated node pairs, each pair of associated nodes in the sequence is used as the two endpoints of the associated relationship edge to build the basic topology of the associated relationship edge between heterogeneous sensing nodes. At the same time, relying on the sequence of multiple sets of semantic parsing results of the associated relationships, the corresponding semantic parsing information is used as the identification basis to accurately identify each associated relationship edge in terms of association type, association strength, semantic attributes, etc., and to give each associated relationship edge a specific feature connotation. Then, according to the temporal features and node association logic, all the identified associated relationship edges are integrated and collected to finally construct multiple sets of associated relationship edges, clearly defining the associated relationship structure and attribute characteristics between heterogeneous sensing nodes in the bank security system.

[0036] In one possible implementation, step S300 further includes: Step S310: According to the K-order association mechanism, multiple heterogeneous sensing nodes are associated with sub-networks from the multi-source behavior sensing network sequence to obtain multiple heterogeneous sensing node multi-source behavior sensing sub-network sequences.

[0037] Step S320: Traverse the multi-source behavior sensing sub-network sequence of the multiple heterogeneous sensing nodes to perform adjacency stability analysis and determine the stability coefficients of the multiple heterogeneous sensing nodes.

[0038] Step S330: Based on the stability coefficients of the multiple heterogeneous sensing nodes, perform risk topology separation on the multi-source behavior sensing network sequence to obtain a consistent relational subgraph sequence and a risk relational subgraph sequence.

[0039] Specifically, based on the K-order association mechanism, which is the association rule of K layers directly or indirectly associated with nodes, the constructed multi-source behavior perception network sequence is subjected to targeted extraction of associated sub-networks. Taking each heterogeneous perception node in the bank security system as the core node, all heterogeneous perception nodes directly or indirectly associated with it within the K-order range, as well as the association edges between these nodes, are mined layer by layer. For each core node, an independent multi-source behavior perception sub-network containing its K-order associated nodes and corresponding association edges is extracted. Then, according to the temporal characteristics of the multi-source behavior perception network sequence, the multi-source behavior perception sub-networks corresponding to all core nodes are temporally integrated to finally obtain multiple heterogeneous perception node multi-source behavior perception sub-network sequences, thereby achieving a refined decomposition of the overall perception network.

[0040] A full traversal is performed on the extracted multi-source behavior-aware sub-network sequences of multiple heterogeneous sensing nodes. The multi-source behavior-aware sub-network sequence corresponding to each heterogeneous sensing node is extracted sequentially as the analysis object. First, a full-dimensional structural feature analysis is performed on the sub-network sequence corresponding to a single node to mine key information such as the sub-network's topology and node connection methods, forming a sub-network structural feature sequence corresponding to that node. Then, adjacency stability analysis is performed on this sub-network structural feature sequence. First, adjacency matrix analysis is completed through fine-grained similarity calculation and normalization to obtain an adjacency matrix sequence. Then, multi-scale approximate identification is performed on the adjacency matrix sequence to obtain a multi-scale approximate similarity set. The stability coefficient of the corresponding heterogeneous sensing node is obtained by calculating the mean of this set. The stability coefficients of all heterogeneous sensing nodes are calculated sequentially according to the above process, and all coefficients are integrated and aggregated to finally determine the stability coefficients of multiple heterogeneous sensing nodes, achieving a quantitative representation of the topological stability state of each node in the network.

[0041] Using the calculated stability coefficients of multiple heterogeneous sensing nodes as the core criterion, the stability coefficient of each heterogeneous sensing node is compared with a preset stability coefficient threshold. If the stability coefficient of a heterogeneous sensing node is greater than or equal to the preset threshold, the node is marked as a consistent node; if its stability coefficient is less than the preset threshold, the node is marked as a risk node. After marking all heterogeneous sensing nodes, all heterogeneous sensing nodes with consistent node markings and their corresponding relational edges are accurately extracted from the original multi-source behavior sensing network sequence. A consistent relational subgraph sequence is constructed according to the original network temporal sequence and topological logic. At the same time, all heterogeneous sensing nodes with risk node markings and their corresponding relational edges are extracted, and a risk relational subgraph sequence is constructed in the same way. This completes the security risk topology separation of the multi-source behavior sensing network sequence, achieving accurate separation of normal relational topology and risk relational topology, and providing targeted topology analysis basis for subsequent dual security abnormal behavior identification.

[0042] In one possible implementation, step S320 further includes: Step S321: Extract the first heterogeneous sensing node multi-source behavior sensing sub-network sequence from the multiple heterogeneous sensing node multi-source behavior sensing sub-network sequences.

[0043] Step S322: Traverse the multi-source behavior perception sub-network sequence of the first heterogeneous sensing node to perform structural feature analysis and obtain the sub-network structural feature sequence.

[0044] Step S323: Perform adjacency stability analysis on the sub-network structure feature sequence to obtain the stability coefficient of the first heterogeneous sensing node.

[0045] Step S324: Add the stability coefficient of the first heterogeneous sensing node to the stability coefficient of multiple heterogeneous sensing nodes.

[0046] Specifically, from the multiple heterogeneous sensing node multi-source behavior sensing sub-network sequences that have been extracted, any one of the heterogeneous sensing node's corresponding multi-source behavior sensing sub-network sequences is selected as the first heterogeneous sensing node multi-source behavior sensing sub-network sequence. The first heterogeneous sensing node is any one of the multiple heterogeneous sensing nodes, not a specific fixed node. After selection, it is used as the first independent analysis unit for adjacency stability analysis.

[0047] A full traversal of the multi-source behavior perception subnetwork sequence of the first heterogeneous sensing node is conducted. To avoid the tediousness of directly analyzing massive behavioral data, the abnormality of node association behavior is judged indirectly by analyzing the changes in subnetwork structure. Specifically, network topology feature extraction is adopted. Each subnetwork in the sequence is analyzed in a fine-grained manner, including node degree distribution, edge connection density, topological clustering coefficient, semantic attribute distribution of associated edges, and adjacency relationship between nodes. At the same time, the temporal change features of the core topological skeleton and associated edges of each subnetwork are extracted. The multi-dimensional structural feature indicators obtained from the analysis of each subnetwork are structured and integrated and arranged in temporal order. Finally, a subnetwork structural feature sequence that can characterize the temporal changes of the structure of the associated subnetwork of the first heterogeneous sensing node is obtained.

[0048] Adjacency stability analysis is performed on the parsed sub-network structure feature sequence to obtain the stability coefficient of the first heterogeneous sensing node. Specifically, adjacency matrix analysis is first performed on the sub-network structure feature sequence. By calculating the fine-grained similarity between two adjacent sub-network structure features in the sequence, a fine-grained similarity set sequence is obtained. Then, each fine-grained similarity set is normalized and filled into an empty matrix to obtain the corresponding adjacency matrix sequence. Next, multi-scale approximation identification is performed on the adjacency matrix sequence to mine the similarity features of the matrix at different scales and form a multi-scale approximate similarity set. Finally, the mean of the multi-scale approximate similarity set is calculated, and the calculation result is used as the stability coefficient of the first heterogeneous sensing node, realizing the quantitative characterization of the adjacency stability state of the node's associated sub-network.

[0049] The addition of stability coefficients for the first heterogeneous sensing node is accomplished using a coefficient aggregation and storage approach. First, an empty set of heterogeneous sensing node stability coefficients is pre-constructed as a coefficient storage carrier. This set is configured with an ordered coefficient input and index mapping mechanism, enabling one-to-one association and binding with heterogeneous sensing nodes. Then, the calculated stability coefficients of the first heterogeneous sensing node are stored in this empty set according to preset coefficient input rules, using numerical writing. Simultaneously, a unique identifier for the corresponding first heterogeneous sensing node is added to each coefficient, completing the association mapping between coefficients and nodes. This allows for the precise addition of the first heterogeneous sensing node's stability coefficients to multiple heterogeneous sensing node stability coefficient sets. This method can then be used to sequentially add and aggregate the stability coefficients of all other heterogeneous sensing nodes.

[0050] In one possible implementation, step S323 further includes: Adjacency matrix analysis is performed on the sub-network structure feature sequence to obtain the adjacency matrix sequence.

[0051] The adjacency matrix sequence is subjected to multi-scale approximate identification to obtain a multi-scale approximate similarity set.

[0052] The mean of the multi-scale approximate similarity set is calculated to obtain the stability coefficient of the first heterogeneous sensing node.

[0053] Specifically, when performing adjacency matrix analysis on the sub-network structure feature sequence to obtain the adjacency matrix sequence, firstly, fine-grained similarity calculation is performed on two temporally adjacent sub-network structure features in the sub-network structure feature sequence. The similarity index between the two is quantified from multiple dimensions such as topology, node connection, and edge attributes to form corresponding fine-grained similarity sets. The fine-grained similarity sets obtained from calculating all adjacent sub-network structure features are integrated in sequence to obtain a fine-grained similarity set sequence. Then, normalization processing is performed on each fine-grained similarity set in the fine-grained similarity set sequence to eliminate differences in data dimensions. The processed values ​​are then sequentially filled into a pre-constructed initially empty matrix. The numerical filling and construction of all matrices are completed in time sequence, and finally, an adjacency matrix sequence that can characterize the changes in the adjacency relationship of the sub-network structure feature sequence is obtained.

[0054] When performing multi-scale approximate identification on the adjacency matrix sequence to obtain a multi-scale approximate similarity set, firstly, multiple levels of scale analysis dimensions are preset, covering different scale levels such as local feature scale, overall topological scale, and temporal change scale of the adjacency matrix. Then, for each adjacency matrix in the adjacency matrix sequence, approximate extraction and matching analysis of matrix features are carried out sequentially at each preset scale. The feature approximation between adjacent adjacency matrices at the same scale is quantitatively calculated. At the same time, the approximate similarity between each adjacency matrix and the reference matrix in the sequence at different scales is calculated. Subsequently, all approximate similarity values ​​calculated at each scale are collected and classified and integrated according to the scale dimension to form an overall multi-scale approximate similarity set. This set can characterize the similarity and change characteristics of the adjacency matrix sequence at different analysis scales in multiple dimensions.

[0055] When calculating the mean of the multi-scale approximate similarity set to obtain the stability coefficient of the first heterogeneous sensing node, the approximate similarity values ​​in the set are first summed to obtain the cumulative sum of all values. Then, the total number of approximate similarity values ​​in the set is counted. Subsequently, the cumulative sum is divided by the total number of values ​​to obtain the arithmetic mean of the multi-scale approximate similarity set. This arithmetic mean is directly used as the stability coefficient of the first heterogeneous sensing node, thereby quantitatively characterizing the adjacency stability of the multi-source behavior sensing sub-network corresponding to the first heterogeneous sensing node. The higher the value, the more stable the topology of the sub-network associated with the node. The lower the value, the worse the stability of the sub-network topology and the higher the possibility of abnormal fluctuations.

[0056] In one possible implementation, step S323 further includes: Fine-grained similarity calculation is performed on two adjacent sub-network structure features in the sub-network structure feature sequence to obtain a fine-grained similarity set sequence.

[0057] Normalize each fine-grained similarity set in the fine-grained similarity set sequence and fill it into an initially empty matrix to obtain an adjacency matrix sequence.

[0058] Specifically, when performing fine-grained similarity calculations on two adjacent sub-network structure features in the sub-network structure feature sequence to obtain a fine-grained similarity set sequence, firstly, pairing analysis is performed on each pair of temporally adjacent sub-network structure features in the sequence. For each pair of adjacent sub-network structure features, the similarity values ​​of the two under each dimension are quantified using algorithms such as feature distance calculation and feature vector matching from multiple fine-grained feature dimensions, such as node degree distribution, topological clustering coefficient, edge connection density, distribution of semantic attributes of associated edges, and node adjacency association. The similarity values ​​of a single pair of adjacent sub-network structure features under all fine-grained dimensions are integrated into a fine-grained similarity set. Then, according to the original temporal order of the sub-network structure feature sequence, the fine-grained similarity sets corresponding to all adjacent feature pairs are arranged sequentially, finally forming a fine-grained similarity set sequence that can characterize the changes in the similarity of adjacent sub-network structure features under multiple dimensions.

[0059] When normalizing and filling matrices in a sequence of fine-grained similarity sets to obtain an adjacency matrix sequence, the extreme value normalization method is first used to uniformly process all fine-grained similarity values ​​within each independent fine-grained similarity set in the sequence, mapping each value to the standard numerical range of [0, 1], eliminating the dimensional differences and numerical range deviations of similarity values ​​under different fine-grained feature dimensions, and ensuring the comparability of similarity values ​​in each dimension. Then, an initial empty matrix matching the fine-grained feature dimensions is pre-constructed. According to the preset row and column index mapping rules, the values ​​in each normalized fine-grained similarity set are sequentially filled into the corresponding row and column positions of the empty matrix, completing the numerical filling and construction of a single matrix. Subsequently, according to the original temporal order of the fine-grained similarity set sequence, all filled matrices are sequentially arranged and integrated to finally obtain an adjacency matrix sequence that can accurately represent the adjacency topology changes of the sub-network structure feature sequence.

[0060] In one possible implementation, step S330 further includes: Step S331: Determine whether the stability coefficient of the plurality of heterogeneous sensing nodes is greater than or equal to the preset stability coefficient threshold. If so, mark the corresponding heterogeneous sensing node as a consistent node.

[0061] Step S332: If not, then identify the corresponding heterogeneous sensing nodes as risk nodes.

[0062] Step S333: Extract heterogeneous sensing nodes with consistent node identifiers from the multi-source behavior perception network sequence, and construct the consistent relation subgraph sequence.

[0063] Step S334: Extract heterogeneous sensing nodes with risk node identifiers from the multi-source behavior perception network sequence and construct the risk relationship subgraph sequence.

[0064] Specifically, for the calculated stability coefficients of multiple heterogeneous sensing nodes, a threshold judgment operation is performed one by one. The stability coefficient value corresponding to each heterogeneous sensing node is compared one-to-one with the system's preset stability coefficient threshold. If the stability coefficient value of a certain heterogeneous sensing node is greater than or equal to the preset stability coefficient threshold, it is determined that the associated sub-network topology of the node is in a stable state with no abnormal fluctuation characteristics. Then, a unique identifier for the consistent node is assigned to the heterogeneous sensing node, completing the labeling operation of the node. This identifier will be bound to the node's unique code, providing a clear node selection basis for the construction of subsequent subgraph sequences.

[0065] After comparing the stability coefficients of multiple heterogeneous sensing nodes with the preset stability coefficient threshold one by one, for heterogeneous sensing nodes with coefficient values ​​less than the preset threshold, it is determined that the stability of the corresponding associated sub-network topology is insufficient and there is a security risk of abnormal fluctuation. Then, a risk node identification operation is performed on such heterogeneous sensing nodes, assigning them a unique risk node label, and binding and storing the label with the node's unique code to form a clear risk node identification file. This provides a clear and unique node screening and identification basis for accurately extracting risk nodes from the multi-source behavior sensing network sequence and constructing a risk relationship subgraph sequence.

[0066] Using the consistent node identifier as the core filtering criterion, a full-domain node retrieval and extraction is performed on the original multi-source behavior perception network sequence. This accurately obtains all heterogeneous perception nodes with consistent node identifiers in the sequence. Simultaneously, all the corresponding relationship edges of these nodes in the multi-source behavior perception network sequence are extracted, along with the original topological connection logic and temporal relationship features between nodes and edges. Subsequently, the original temporal arrangement and topological structure of these nodes and relationship edges are preserved. According to the original temporal dimension of the multi-source behavior perception network sequence, the extracted consistent nodes and corresponding relationship edges are re-integrated and reconstructed into a sub-network sequence, ultimately forming a consistent relationship subgraph sequence. This subgraph sequence fully preserves the topological relationship and temporal change features of stable nodes in the original network.

[0067] Using the risk node identifiers as the core retrieval criteria, a global node and topology extraction operation is performed on the original multi-source behavior perception network sequence. This accurately filters out all heterogeneous perception nodes with risk node identifiers in the sequence, and simultaneously extracts all the corresponding relational edges of these nodes in the original network sequence, as well as the inherent topological connection logic and temporal relational features between nodes and edges. Subsequently, the original temporal arrangement and topological connection relationship of the extracted risk nodes and relational edges are fully preserved. Following the original temporal dimension of the multi-source behavior perception network sequence, the extracted risk nodes and their corresponding relational edges are re-integrated and reconstructed into a sub-network sequence, ultimately forming a risk relational subgraph sequence. This subgraph sequence fully preserves the topological relational and temporal change features of the risk nodes in the original network.

[0068] In one possible implementation, step S400 further includes: Step S410: Call the conventional anomaly recognition network to perform security anomaly behavior recognition on the consistency relation subgraph sequence and obtain the first anomaly behavior recognition result.

[0069] Step S420: Call the risk anomaly identification network to identify security anomalies in the risk relationship subgraph sequence and obtain the second anomaly identification result.

[0070] Step S430: Summarize the first abnormal behavior identification results and the second abnormal behavior identification results to obtain the abnormal behavior identification results.

[0071] Specifically, a conventional anomaly recognition network based on CNN-LSTM fusion is invoked to perform security anomaly behavior recognition on the consistency relation subgraph sequence to obtain the first anomaly behavior recognition result. The specific implementation is as follows: First, the consistency relation subgraph sequence is divided into fixed-length topological feature blocks according to time sequence. The spatial topological features such as the node adjacency structure and semantic attributes of associated edges of each feature block are extracted through CNN convolutional layers to obtain multi-dimensional spatial feature vectors. Then, the temporally continuous spatial feature vectors are input into LSTM recurrent layers to capture the temporal change patterns of security behavior under stable topology in the subgraph sequence. The spatiotemporal fusion features are reduced in dimensionality and classified through fully connected layers. Finally, combined with the bank's conventional security anomaly behavior label library, which includes labels such as internal personnel violations, abnormal lingering in areas, and conventional security equipment malfunctions, the abnormal behavior category, confidence level, and occurrence time of each time segment are output through a Softmax classifier. All recognition results are integrated according to the original sequence time sequence. At the same time, the node source and location of anomaly results with confidence levels higher than a preset threshold are traced and located. Finally, the first anomaly behavior recognition result containing the anomaly behavior type, occurrence location, time sequence information, confidence level, and associated topological nodes is formed.

[0072] A risk anomaly identification network based on GAT-GRU fusion is invoked to identify security anomalies in risk relationship subgraph sequences to obtain a second anomaly identification result. First, the risk relationship subgraph sequence is topologically encoded. Then, a GAT graph attention layer is used to extract the adjacency association features and topological fluctuation features of risk nodes with weights, focusing on the core features of high-risk nodes and associated edges to generate node attention feature vectors, fully exploring the nonlinear associations and structural anomalies between nodes in the risk subgraph. Next, the temporally sequenced node attention feature vectors are input into a GRU gated recurrent layer to capture the dynamic mutation patterns and temporal anomalies of security behaviors in the risk subgraph sequence, suppressing interference from ineffective and redundant features. Finally, a fully connected layer completes the temporal anomaly identification. The spatial fusion features are transformed and mapped, combined with a high-risk abnormal behavior label library for bank security, including labels such as abnormal internal personnel movement, emerging fire hazards, abnormal security equipment malfunctions, and illegal intrusion into outdoor areas. A sigmoid multi-classifier is used to output the abnormal behavior category, risk level, probability of occurrence, and associated risk nodes for each time segment. Finally, all identification results are integrated in time according to the original risk relationship subgraph sequence. The abnormal results with a probability of occurrence higher than a preset threshold are subjected to topological tracing and spatial localization. At the same time, the propagation trend and potential impact range of abnormal behavior are marked. Finally, a second abnormal behavior identification result is formed, which includes abnormal behavior type, risk level, occurrence location and time sequence, confidence level, associated risk nodes, and propagation characteristics.

[0073] The first and second abnormal behavior identification results are standardized, integrated, and fused to obtain the final abnormal behavior identification result. First, the data formats of the two types of results are standardized. Core information such as abnormal behavior type, spatiotemporal location, associated heterogeneous sensing nodes, anomaly confidence level, and risk level are reconstructed according to the unified coding rules of the bank's security system, eliminating differences in data dimensions and identification rules between the two types of identification results. Next, an information matching algorithm is used to check for duplicates of the same security anomaly. For duplicate identifications of the same security anomaly, the more accurate confidence level and risk level are retained, while non-duplicate anomaly information is merged across all dimensions. Then, all integrated anomaly information is screened for validity, eliminating invalid identification results with confidence levels below the system's preset threshold. Finally, valid anomaly information is classified, sorted, and structured according to the bank's security area division, anomaly risk level, and the order of occurrence. Corresponding handling prompts for the anomaly are added, ultimately forming a complete abnormal behavior identification result that includes anomaly behavior type, precise spatiotemporal location, associated heterogeneous sensing nodes and topological relationships, comprehensive confidence level, risk level, and handling priority.

[0074] Example 2, based on the same inventive concept as the bank security anomaly behavior identification method based on multimodal data in the previous examples, such as... Figure 2 As shown, this application provides a bank security abnormal behavior recognition system based on multimodal data. The system and method embodiments in this application are based on the same inventive concept. The system includes: The heterogeneous sensing node acquisition module 10 is used to acquire multiple heterogeneous sensing nodes of the bank security system and deploy multimodal sensing modules at the multiple heterogeneous sensing nodes respectively.

[0075] The multi-source behavior perception module 20 is used to traverse the multiple heterogeneous perception nodes within a preset window to perform multi-source behavior perception and obtain a multi-source behavior perception network sequence.

[0076] The security risk topology separation module 30 is used to traverse the multi-source behavior perception network sequence to perform security risk topology separation and obtain a consistent relationship subgraph sequence and a risk relationship subgraph sequence.

[0077] The abnormal behavior identification result acquisition module 40 is used to perform dual security abnormal behavior identification based on the consistency relationship subgraph sequence and the risk relationship subgraph sequence to obtain abnormal behavior identification results.

[0078] The collaborative verification module 50 is used to adaptively schedule the real-time security patrol strategy based on the abnormal behavior identification results, obtain the scheduling security handling instructions, and conduct unmanned vehicle or drone collaborative verification of the bank security system based on the scheduling security handling instructions.

[0079] Furthermore, the system is also used for the following functions: The behavior of the multiple heterogeneous sensing nodes within a preset window is perceived by the multimodal sensing module to obtain multiple heterogeneous sensing node behavior data sequences; based on the multiple heterogeneous sensing node behavior data sequences, the correlation analysis between the multiple heterogeneous sensing nodes is performed to determine multiple sets of correlation edge sequences; based on the multiple heterogeneous sensing nodes and multiple sets of correlation edge sequences, a multi-source sensing network sequence is constructed.

[0080] Furthermore, the system is also used for the following functions: Multiple heterogeneous sensing nodes are used as source nodes, and associated nodes are extracted from the behavioral data sequences of the multiple heterogeneous sensing nodes to obtain multiple sets of associated node pairs. Using the multiple sets of associated node pairs as indexes, semantic parsing of the association relationships is performed on the behavioral data sequences of the multiple heterogeneous sensing nodes to determine multiple sets of ...

[0081] Furthermore, the system is also used for the following functions: According to the K-order association mechanism, multiple heterogeneous sensing nodes are extracted from the multi-source behavior sensing network sequence to obtain multiple heterogeneous sensing node multi-source behavior sensing sub-network sequences; adjacency stability analysis is performed on the multiple heterogeneous sensing node multi-source behavior sensing sub-network sequences to determine the stability coefficients of multiple heterogeneous sensing nodes; based on the stability coefficients of multiple heterogeneous sensing nodes, risk topology separation is performed on the multi-source behavior sensing network sequence to obtain a consistency relationship sub-graph sequence and a risk relationship sub-graph sequence.

[0082] Furthermore, the system is also used for the following functions: Extract the first heterogeneous sensing node multi-source behavior sensing sub-network sequence from the multiple heterogeneous sensing node multi-source behavior sensing sub-network sequences; traverse the first heterogeneous sensing node multi-source behavior sensing sub-network sequence to perform structural feature analysis and obtain the sub-network structural feature sequence; perform adjacency stability analysis on the sub-network structural feature sequence to obtain the first heterogeneous sensing node stability coefficient; add the first heterogeneous sensing node stability coefficient to the multiple heterogeneous sensing node stability coefficients.

[0083] Furthermore, the system is also used for the following functions: Adjacency matrix analysis is performed on the sub-network structure feature sequence to obtain an adjacency matrix sequence; multi-scale approximate identification is performed on the adjacency matrix sequence to obtain a multi-scale approximate similarity set; the mean of the multi-scale approximate similarity set is calculated to obtain the stability coefficient of the first heterogeneous sensing node.

[0084] Furthermore, the system is also used for the following functions: Fine-grained similarity calculation is performed on two adjacent sub-network structure features in the sub-network structure feature sequence to obtain a fine-grained similarity set sequence; each fine-grained similarity set in the fine-grained similarity set sequence is normalized and filled into an initially empty matrix to obtain an adjacency matrix sequence.

[0085] Furthermore, the system is also used for the following functions: Determine whether the stability coefficients of the multiple heterogeneous sensing nodes are greater than or equal to a preset stability coefficient threshold. If yes, mark the corresponding heterogeneous sensing node as a consistent node; otherwise, mark the corresponding heterogeneous sensing node as a risk node. Extract the heterogeneous sensing nodes with consistent node markers from the multi-source behavior sensing network sequence and construct the consistency relationship subgraph sequence. Extract the heterogeneous sensing nodes with risk node markers from the multi-source behavior sensing network sequence and construct the risk relationship subgraph sequence.

[0086] Furthermore, the system is also used for the following functions: The conventional anomaly identification network is invoked to identify security anomalies in the consistency relation subgraph sequence, and a first anomaly identification result is obtained; the risk anomaly identification network is invoked to identify security anomalies in the risk relation subgraph sequence, and a second anomaly identification result is obtained; the first anomaly identification result and the second anomaly identification result are combined to obtain the anomaly identification result.

[0087] It should be noted that the order of the embodiments described above is for descriptive purposes only and does not represent the superiority or inferiority of the embodiments. Specific embodiments of this specification have been described above. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0088] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0089] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application intends to include such modifications and variations.

Claims

1. A method for identifying abnormal bank security behavior based on multimodal data, characterized in that, The method includes: Acquire multiple heterogeneous sensing nodes of the bank's security system, and deploy multimodal sensing modules at each of the multiple heterogeneous sensing nodes; The multiple heterogeneous sensing nodes are traversed within a preset window to perform multi-source behavior sensing, thereby obtaining a multi-source behavior sensing network sequence. The security risk topology is separated by traversing the multi-source behavior perception network sequence to obtain a consistent relationship subgraph sequence and a risk relationship subgraph sequence. Based on the consistency relationship subgraph sequence and the risk relationship subgraph sequence, dual security abnormal behavior identification is performed to obtain the abnormal behavior identification result; Based on the abnormal behavior identification results, the real-time security patrol strategy is adaptively scheduled to obtain the scheduling security handling instructions. Based on the scheduling security handling instructions, the bank's security system is used for unmanned vehicle or drone collaborative verification.

2. The bank security abnormal behavior identification method based on multimodal data as described in claim 1, characterized in that, Traversing the multiple heterogeneous sensing nodes within a preset window to perform multi-source behavior sensing, a multi-source behavior sensing network sequence is obtained, including... The behavior of the multiple heterogeneous sensing nodes within a preset window is perceived by the multimodal sensing module, thereby obtaining a sequence of behavior data of the multiple heterogeneous sensing nodes. Based on the behavioral data sequences of the multiple heterogeneous sensing nodes, an association analysis is performed between the multiple heterogeneous sensing nodes to determine multiple sets of association relationship edges. A multi-source sensing network sequence is constructed based on the multiple heterogeneous sensing nodes and multiple sets of associated edge sets.

3. The bank security abnormal behavior identification method based on multimodal data as described in claim 2, characterized in that, Based on the behavioral data sequences of the multiple heterogeneous sensing nodes, an association analysis is performed between the multiple heterogeneous sensing nodes to determine multiple sets of association edge sequences, including: Using multiple heterogeneous sensing nodes as source nodes, the associated nodes in the behavioral data sequences of the multiple heterogeneous sensing nodes are extracted to obtain a set sequence of multiple associated node pairs. Using the sequence of multiple associated node pairs as an index, semantic parsing of the association relationships is performed on the sequence of behavioral data of multiple heterogeneous sensing nodes to determine a sequence of multiple sets of semantic parsing results of association relationships. Each associated node pair in the sequence of multiple associated node pairs is used as the two endpoints of the associated relationship edge, and the associated relationship edge is identified using the sequence of associated relationship semantic parsing results, thereby constructing the sequence of multiple associated relationship edge sets.

4. The bank security abnormal behavior identification method based on multimodal data as described in claim 1, characterized in that, The security risk topology is separated by traversing the multi-source behavior-aware network sequence to obtain a consistent relational subgraph sequence and a risk relational subgraph sequence, including: According to the K-order association mechanism, multiple heterogeneous sensing nodes are associated subnetworks extracted from the multi-source behavior sensing network sequence to obtain multiple heterogeneous sensing node multi-source behavior sensing subnetwork sequences. Adjacency stability analysis is performed by traversing the multi-source behavior sensing sub-network sequence of the multiple heterogeneous sensing nodes to determine the stability coefficients of the multiple heterogeneous sensing nodes. Based on the stability coefficients of the multiple heterogeneous sensing nodes, risk topology separation is performed on the multi-source behavior sensing network sequence to obtain a consistent relational subgraph sequence and a risk relational subgraph sequence.

5. The bank security abnormal behavior identification method based on multimodal data as described in claim 4, characterized in that, Adjacency stability analysis is performed on the multi-source behavior sensing sub-network sequence of the multiple heterogeneous sensing nodes to determine the stability coefficients of the multiple heterogeneous sensing nodes, including: Extract the first heterogeneous sensing node multi-source behavior sensing sub-network sequence from the multiple heterogeneous sensing node multi-source behavior sensing sub-network sequences; The structural features of the subnetwork are analyzed by traversing the multi-source behavior perception subnetwork sequence of the first heterogeneous sensing node to obtain the subnetwork structural feature sequence. Adjacency stability analysis is performed on the sub-network structure feature sequence to obtain the stability coefficient of the first heterogeneous sensing node; The stability coefficient of the first heterogeneous sensing node is added to the stability coefficient of multiple heterogeneous sensing nodes.

6. The method for identifying abnormal bank security behavior based on multimodal data as described in claim 5, characterized in that, Adjacency stability analysis is performed on the sub-network structure feature sequence to obtain the stability coefficient of the first heterogeneous sensing node, including: Adjacency matrix analysis is performed on the sub-network structure feature sequence to obtain the adjacency matrix sequence; The adjacency matrix sequence is subjected to multi-scale approximate identification to obtain a multi-scale approximate similarity set; The mean of the multi-scale approximate similarity set is calculated to obtain the stability coefficient of the first heterogeneous sensing node.

7. The bank security abnormal behavior identification method based on multimodal data as described in claim 6, characterized in that, Adjacency matrix analysis is performed on the sub-network structure feature sequence to obtain an adjacency matrix sequence, including: Fine-grained similarity calculation is performed on two adjacent sub-network structure features in the sub-network structure feature sequence to obtain a fine-grained similarity set sequence; Normalize each fine-grained similarity set in the fine-grained similarity set sequence and fill it into an initially empty matrix to obtain an adjacency matrix sequence.

8. The method for identifying abnormal bank security behavior based on multimodal data as described in claim 1, characterized in that, Based on the stability coefficients of the multiple heterogeneous sensing nodes, risk topology separation is performed on the multi-source behavior sensing network sequence to obtain a consistent relational subgraph sequence and a risk relational subgraph sequence, including: Determine whether the stability coefficient of the plurality of heterogeneous sensing nodes is greater than or equal to a preset stability coefficient threshold. If so, mark the corresponding heterogeneous sensing node as a consistent node. If not, the corresponding heterogeneous sensing nodes will be identified as risk nodes; Extract heterogeneous sensing nodes with consistent node identifiers from the multi-source behavior perception network sequence, and construct the consistent relationship subgraph sequence; Extract heterogeneous sensing nodes with risk node identifiers from the multi-source behavior perception network sequence, and construct the risk relationship subgraph sequence.

9. The method for identifying abnormal bank security behavior based on multimodal data as described in claim 1, characterized in that, Based on the consistency relation subgraph sequence and the risk relation subgraph sequence, dual security anomaly behavior identification is performed to obtain anomaly behavior identification results, including: A conventional anomaly detection network is invoked to identify security anomalies in the consistency relation subgraph sequence, and a first anomaly detection result is obtained. The risk anomaly identification network is invoked to identify security anomalies in the risk relationship subgraph sequence, and a second anomaly identification result is obtained. The abnormal behavior identification results are obtained by summing the first abnormal behavior identification results and the second abnormal behavior identification results.

10. A bank security abnormal behavior recognition system based on multimodal data, characterized in that, The system is used to implement the bank security abnormal behavior identification method based on multimodal data as described in any one of claims 1-9, and the system includes: The heterogeneous sensing node acquisition module is used to acquire multiple heterogeneous sensing nodes of the bank security system and deploy multimodal sensing modules at each of the multiple heterogeneous sensing nodes. The multi-source behavior perception module is used to traverse the multiple heterogeneous perception nodes within a preset window to perform multi-source behavior perception and obtain a multi-source behavior perception network sequence. The security risk topology separation module is used to traverse the multi-source behavior perception network sequence to perform security risk topology separation and obtain a consistent relationship subgraph sequence and a risk relationship subgraph sequence. The abnormal behavior identification result acquisition module is used to perform dual security abnormal behavior identification based on the consistency relationship subgraph sequence and the risk relationship subgraph sequence to obtain the abnormal behavior identification result. The collaborative verification module is used to adaptively schedule the real-time security patrol strategy based on the abnormal behavior identification results, obtain the scheduling security handling instructions, and conduct unmanned vehicle or drone collaborative verification of the bank's security system based on the scheduling security handling instructions.

Images