Internal control risk dynamic identification method and system based on multi-modal model

Abstract

This invention discloses a method and system for dynamic identification of internal control risks based on a multimodal model, belonging to the technical field of dynamic risk identification methods. This invention utilizes an unsupervised state transition analysis approach to dynamically identify hidden abnormal transition segments within behavioral patterns, eliminating the need for manually set rules and enhancing the adaptive capability of risk identification. When multimodal logs exhibit time lag, an operation chain backtracking and virtual calibration mechanism ensures the temporal consistency and causal integrity of cross-modal behavioral chains, significantly reducing the probability of misjudgments due to inconsistent records. By employing a sliding matching method to map state sequences to a process topology graph, it effectively identifies structurally mismatched fragments, accurately locating the start and end points of risky behaviors.

Description

Technical Field

[0001] This invention relates to the field of dynamic risk identification methods, and in particular to a method and system for dynamic identification of internal control risks based on a multimodal model. Background Technology

[0002] With the continuous integration of information technology and internal control management systems, internal control risk identification methods based on big data, artificial intelligence, and other technologies have become important supporting tools for organizational governance. In recent years, researchers both domestically and internationally have conducted extensive research and practical exploration on the structural modeling, semantic understanding, and risk detection mechanisms of internal control data, particularly in data-intensive fields such as administration, healthcare, and finance. By constructing neural network models, rule engines, and logical reasoning frameworks, they have significantly improved data utilization efficiency and risk response capabilities by perceiving and dynamically monitoring risks in internal processes. However, existing technologies still face challenges in addressing internal control operations, including weak multimodal data collaboration, a lack of temporal rhythm modeling for state change sequences, and difficulty in accurately capturing structural deviation characteristics. These limitations make it difficult to meet the real-time, accurate, and structurally complete requirements for risk identification in complex business processes.

[0003] CN119989421A proposes a method and system for processing internal control information in administrative and public institutions based on big data analysis. By employing neural networks to extract semantic embedding features from business domains and information content, and establishing a context-aware cross-dimensional joint coding mechanism, it achieves dynamic correlation modeling and sensitivity judgment between business semantics, demonstrating practical value in information desensitization and data leakage prevention. However, this solution focuses more on the joint representation and chain-like logical relationship modeling at the static semantic level, failing to consider the dynamic rhythmic characteristics of data at different time scales. Especially when facing complex multi-stage operational processes, it lacks effective perception and structural calibration methods for micro-behavioral anomalies such as time lag and operational breaks. Therefore, it is difficult to achieve highly sensitive identification of rhythmic anomalies and structural drift phenomena in the process, affecting the timeliness and accuracy of risk warnings.

[0004] CN119625784A discloses an automated early warning system and method for hospital risk internal control based on big data. It focuses on analyzing crowd behavior trajectory characteristics through video surveillance and image processing to achieve feedback identification of abnormal behavior in hospital corridors. This method effectively improves target monitoring capabilities under dynamic occlusion and establishes a behavioral early warning model through trajectory features and feedback values, making it suitable for risk detection in specific physical scenarios. However, it is primarily image-based, relying on visual perception and trajectory matrices for risk inference, making it difficult to apply to complex process internal control environments involving the collaborative processing of abstract data modalities such as logs, text, and forms. Furthermore, its risk detection logic does not cover historical state tracing and structural consistency verification of the operation chain, failing to deeply model and identify potential operational deviations and mismatch risks at the data dimension and process topology level. Summary of the Invention

[0005] In view of the problems existing in the identification of internal control risks based on semantic modeling or behavioral trajectory perception when facing risk identification in the face of multimodal data interaction and multi-stage operation process, this invention is proposed.

[0006] Therefore, the problem to be solved by this invention is how to extract multimodal shared feature vectors by fusing attention mechanisms, and combine unsupervised state transition modeling and operation chain calibration techniques to achieve accurate identification of rhythm anomalies, time lags and structurally isolated fragments.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] In a first aspect, this invention provides a method for dynamic identification of internal control risks based on a multimodal model. This method includes: extracting a set of feature vectors representing shared semantics from data across multiple modalities using a fusion attention mechanism, and serializing the time dimension to obtain an internal control operation sequence tensor; performing unsupervised state transition analysis on the internal control operation sequence tensor to identify rhythmic anomalies in multi-stage operations with intervals lower than the global expected window, and constructing a state transition graph; based on the state transition graph, when a timestamp lag is found in the log modality corresponding to any transition path, activating operation chain backtracking, inserting a virtual calibration node and attaching a label; mapping the calibrated state transition graph onto a preset process topology graph, performing subgraph sliding matching of length n, and if n consecutive time-series state nodes cannot form a connected subgraph in the process topology graph, they are determined to be structurally isolated fragments, and the start and end node indices are output as risk warning points, where n is a constant.

[0009] As a preferred embodiment of the dynamic identification method for internal control risks based on multimodal models described in this invention, the extraction of the multimodal shared semantic feature vector set includes: using the timestamp field as a benchmark, aligning the original data of each modality with a fixed-length sliding window strategy to construct a synchronous data block sequence, and removing segments with cross-modal time window overlap rates below a set threshold; inputting the synchronous data block sequence into a cross-modal attention network to identify the dominant contribution of key modalities to behavioral features within the current time window at the semantic level, and generating a shared semantic tensor accordingly to suppress weakly correlated modal noise.

[0010] As a preferred embodiment of the dynamic identification method for internal control risks based on a multimodal model as described in this invention, the unsupervised state transition analysis includes: segmenting the internal control operation sequence tensor into windows along the time dimension; calculating the operation state cluster center within each segment using an adaptive clustering distance function to generate a time-series state node set; and constructing a state transition candidate graph in an unsupervised manner based on the time-series state node set, wherein each edge has a weight set according to the operation order and behavioral variability between nodes.

[0011] As a preferred embodiment of the dynamic identification method for internal control risks based on a multimodal model described in this invention, the construction of the state transition graph includes: performing rhythmic coherence constraint detection on the state transition candidate graph according to the global expected window length, marking any edge whose state transition interval is less than the global expected window as an abnormal transition edge; performing subgraph pruning on the state transition candidate graph containing abnormal transition edges, removing segments with continuous non-abnormal edges that are less than the pruning threshold, and outputting a structured state transition graph.

[0012] As a preferred embodiment of the dynamic identification method for internal control risks based on a multimodal model described in this invention, the activation operation chain backtracking includes: based on the state transition graph, extracting the log modality timestamp sequence associated with each abnormal transition edge, aligning it with the timestamp field, calculating the cross-modal lag matrix, and identifying lag transition paths with time lags higher than the ratio threshold; for lag transition paths, recursively searching forward from the starting node to extract the shortest path segment with continuous abnormal transition edges as the backtracking sub-chain.

[0013] As a preferred embodiment of the dynamic identification method for internal control risks based on multimodal models described in this invention, the step of inserting virtual calibration nodes and attaching tags includes: determining the position of the weakest modality segment in the backtracking subchain, inserting a virtual calibration node with a time interpolation marker, and attaching a corresponding modality type label and a delay estimate to the node; mapping the backtracking subchain structure after inserting the virtual calibration node back to and updating the state transition graph to form a calibrated state transition graph.

[0014] As a preferred embodiment of the dynamic identification method for internal control risks based on multimodal models described in this invention, the step of mapping the calibrated state transition graph to a preset process topology graph includes: extracting time-series state nodes arranged in chronological order from the calibrated state transition graph, and constructing a set of sliding sub-paths consisting of n consecutive state nodes using a sliding window of fixed length n; for each sliding sub-path, sequentially searching for paths in the process structure graph where the node order is consistent and any adjacent nodes are directly connected by directed edges in the process structure graph.

[0015] Secondly, this invention provides a dynamic identification system for internal control risks based on a multimodal model, comprising: a semantic extraction module, which extracts a set of feature vectors for shared semantics from data of multiple modalities using a fusion attention mechanism, and performs serialization processing on the time dimension to obtain an internal control operation sequence tensor; a state analysis module, which performs unsupervised state transition analysis on the internal control operation sequence tensor to identify rhythmic anomalies in multi-stage operations with intervals lower than the global expected window, and constructs a state transition graph; a backtracking calibration module, which, based on the state transition graph, activates operation chain backtracking, inserts virtual calibration nodes and attaches tags when a timestamp lag is found in the log modality corresponding to any transition path; and a structure detection module, which maps the calibrated state transition graph to a preset process topology graph, performs subgraph sliding matching of length n, and if n consecutive time series state nodes cannot form a connected subgraph in the process topology graph, it is determined to be a structural free segment and outputs the start and end node indices as risk warning points, where n is a constant.

[0016] Thirdly, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein the computer program instructions, when executed by the processor, implement the steps of the dynamic identification method for internal control risks based on a multimodal model as described in the first aspect of the present invention.

[0017] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program instructions are executed by a processor, they implement the steps of the dynamic identification method for internal control risks based on a multimodal model as described in the first aspect of the present invention.

[0018] The beneficial effects of this invention are as follows: Through unsupervised state transition analysis, this invention can dynamically identify hidden abnormal transition segments in behavioral patterns without relying on manually set rules, thus enhancing the adaptive capability of risk identification. When multimodal logs exhibit time lag issues, the invention ensures the temporal consistency and causal integrity of cross-modal behavioral chains through operation chain backtracking and virtual calibration mechanisms, significantly reducing the probability of misjudgments due to record inconsistencies. By employing a sliding matching method to map state sequences to the process topology graph, it effectively identifies structurally mismatched fragments and can accurately locate the start and end points of risky behaviors. In summary, this invention possesses high adaptability, high robustness, and high interpretability, providing strong technical support for the dynamic identification of internal control risks. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a method for dynamic identification of internal control risks based on a multimodal model.

[0021] Figure 2 This is a structural diagram of an internal control risk dynamic identification system based on a multimodal model. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] As mentioned in the background section, in recent years, researchers both domestically and internationally have conducted extensive research and practical exploration on the structural modeling, semantic understanding, and risk detection mechanisms of internal control data. This is particularly evident in data-intensive fields such as administration, healthcare, and finance. By constructing neural network models, rule engines, and logical reasoning frameworks, researchers have significantly improved data utilization efficiency and risk response capabilities by enabling risk perception and dynamic monitoring of internal processes. However, existing technologies still face challenges in addressing internal control operations, including weak multimodal data collaboration, a lack of temporal rhythm modeling for state change sequences, and difficulty in accurately capturing structural deviation features. Consequently, they fall short of meeting the real-time, accurate, and structurally complete requirements for risk identification in complex business processes.

[0026] Figure 1 A flowchart of a dynamic identification method for internal control risks based on a multimodal model according to an embodiment of this application is shown. (Refer to...) Figure 1 As shown, the internal control risk dynamic identification method based on a multimodal model in this embodiment of the invention includes:

[0027] S1: Extract the set of semantic feature vectors shared by multiple modalities from the data using a fusion attention mechanism, and serialize the time dimension to obtain the internal control operation sequence tensor.

[0028] First, for approval logs, behavior trajectories, operation timestamps, and video surveillance data, a modal index set is constructed based on the collection source, structural dimension, and timestamp field. Using the timestamp field in the modal index set as a benchmark, a fixed-length sliding window strategy is adopted to align the original data of each modality, construct a synchronous data block sequence, and remove segments with cross-modal time window overlap rates below a set threshold.

[0029] Traditional methods often use timestamps for approximate alignment, but this approach is prone to significant time drift or loss of modal information in asynchronously acquired data scenarios. This invention calculates the overlap of modal data within each synchronous data block. If the data coverage ratio of any modality within a given time window is found to be lower than a preset value (e.g., 40%), the time window is automatically marked as a low-confidence window and removed. This operation not only improves the robustness of subsequent semantic fusion but also effectively avoids feature bias caused by missing modal data.

[0030] Furthermore, the sequence of synchronized data blocks is input into a cross-modal attention network to identify the dominant contribution of key modalities to behavioral features within the current time window at the semantic level, and a shared semantic tensor is generated accordingly to suppress weakly correlated modal noise.

[0031] In practice, approval logs are first converted into text embedding vectors, behavioral trajectories are converted into displacement pattern vectors, and video frames are extracted into visual feature vectors, all of which are then uniformly encoded into the same feature space. Subsequently, based on the intermodal attention interaction mechanism, the modal combination that dominates behavioral information within a certain time window is automatically identified, thereby generating a shared semantic tensor that suppresses the interference effects of irrelevant or weakly correlated modalities to the greatest extent possible.

[0032] Finally, the shared semantic tensor is input into a temporal awareness network with a residual gating structure to preserve behavioral pattern changes across time periods, and the final output is an internal control operation sequence tensor.

[0033] It should be noted that, due to the temporal variation of user behavior, analyzing each window in isolation would fail to reveal the behavioral evolution trend of risk events. Therefore, this invention, to achieve sequential modeling of shared semantic tensors in the time dimension, inputs the shared semantic tensors into a time-series-aware network with a residual gating structure. This time-series-aware network integrates bidirectional gated recurrent units (Bi-GRUs) and residual connection modules. On the one hand, it captures the evolutionary characteristics of behavioral patterns over time; on the other hand, it preserves the original feature information through the residual mechanism, avoiding the gradient vanishing problem caused by deep time-series modeling. In this network, each shared semantic tensor is treated as a time step vector in the time series, and multiple time windows are concatenated to form the input sequence. The time-series-aware network captures the dependencies between time steps through a bidirectional propagation mechanism, thereby enhancing the model's ability to express complex behavioral pattern changes. The final output of the network is the internal control operation sequence tensor, which maintains the semantic expression of behavior while possessing good temporal continuity and context awareness.

[0034] S2: Perform unsupervised state transition analysis on the internal control operation sequence tensor to identify rhythmic anomalies in multi-stage operations with intervals lower than the global expected window, and construct a state transition graph.

[0035] S2.1: The internal control operation sequence tensor is segmented into windows along the time dimension. Within each segment, the center of the operation state cluster is calculated using an adaptive clustering distance function to generate a set of time series state nodes.

[0036] Specifically, the segmentation process involves extracting local operational features within each global expectation window and dividing them into sub-sequences. The step size of the sliding window is set to half the length of the global expectation window to ensure overlap between segments, thereby improving the coherence of sequence connection and contextual consistency.

[0037] The adaptive clustering distance function not only considers the Euclidean distance between semantic vectors but also sets a modality weight adjustment term, giving higher weights to feature dimensions with higher modality confidence in the semantic vectors during clustering. The advantage of this design is that it gives greater influence to highly relevant modalities (such as operation logs) and suppresses the interference of low-quality modalities (such as some blurred video frames) on the clustering results. Ultimately, each time period will yield a cluster center node representing the state center, forming a time-series state node set.

[0038] S2.2: Based on the time-series set of state nodes, an unsupervised method is used to construct a candidate graph for state transitions. Each edge is weighted according to the order of operations between nodes and the variability of behavior, and the influence of multimodal heterogeneity is balanced according to the entropy weight coefficient. The specific operation steps are as follows:

[0039] S2.2.1: Based on the generated time-series set of state nodes, construct the initial state connection path according to the time order of the state nodes in the operation sequence, which serves as the backbone structure of the candidate state transition graph.

[0040] In this graph, nodes in the state transition candidate graph correspond to the centers of various state clusters, and edges represent jump behaviors in the operation sequence. Under unsupervised conditions, the nodes are first sorted according to their timestamps, with each node's time being t. i If t i <t i+1 Then, an edge is constructed.

[0041] S2.2.2: For any pair of adjacent state nodes, set the edge weight for state transition based on the weighted Mahalanobis distance metric, highlighting the transition edges with low behavioral similarity.

[0042] It should be noted that the edge weight calculation is based on the weighted Mahalanobis distance function, which can better capture behavioral shifts under the covariance structure. The larger the calculated result, the more significant the difference in behavioral characteristics between the two states, and the more likely these edges should be prioritized for detection. This mechanism can automatically amplify the behavioral differences between potentially anomalous operations.

[0043] S2.2.3: In the state transition candidate graph, redundant edges with edge weights lower than the set threshold are removed, and state jump paths with high information gain are retained to enhance the boundary clarity of subsequent rhythm anomaly detection.

[0044] To avoid overly complex graph structures, a minimum edge weight threshold is set. Edges with weights less than this threshold are considered low-information-gain paths and are deleted. This step not only simplifies the size of the state graph but also improves the efficiency and accuracy of subsequent graph traversals, and helps to clearly identify the boundaries of edges with abnormal rhythms.

[0045] S2.3: Based on the global expected window length, perform rhythmic coherence constraint detection on the state transition candidate graph, and mark any edge whose state transition interval is less than the expected window as an abnormal transition edge, including the following operations:

[0046] S2.3.1: From the generated state transition candidate graph, extract all state transition edges with direct connections, record the time interval corresponding to each edge, that is, the time step difference between the source node and the target node in the original time series, and construct a set of transition time intervals.

[0047] S2.3.2: Based on the global expected window length parameter Δτ, compare each edge of the jump time interval set, filter out all jump edges that are less than Δτ, and mark them in the figure.

[0048] Specifically, the process for setting the global expected window length parameter Δτ is as follows:

[0049] By statistically analyzing the center time intervals of adjacent data blocks in the synchronized data block sequence, a rhythm interval set is formed. By performing mean or median statistics on the rhythm interval set, the global expected window length parameter Δτ can be obtained, which is used to characterize the average rhythm level of modal synchronization behavior in the current task scenario.

[0050] S2.3.3: For edges marked as rhythmic aberrations, further differentiated level labels are assigned to aberration edges based on the edge weight strength of the connecting nodes and the node modal labels, in order to enhance the stability of boundary recognition in subsequent structural trimming.

[0051] The determination of the differential level of abnormal edges includes: first, calculating the average and standard deviation of all edge weights; for each edge marked as a rhythmic abnormal jump, judging the difference between the edge weight and the average edge weight; if the edge weight is significantly higher than the average, it indicates that the edge is structurally abnormally strong, possibly representing an abnormally high-frequency jump; if the edge weight is much lower than the average, it indicates that the edge may be structurally abnormally weak, possibly representing a discontinuous mutation; if the two nodes come from different modalities, it indicates that the edge has a cross-modal jump, the information flow stability is poor, and vigilance is required; if the two nodes come from the same modality, it indicates that the jump is an abnormal connection within the modality, and the stability is relatively strong; if an edge weight is significant (i.e., deviates significantly from the mean) and the connected modalities are inconsistent, the corresponding edge should be given a high-level abnormality label; if the edge weight does not change much and the modality is consistent, the corresponding edge is considered to have a low-level abnormality; if only one of them is significant, it is given a medium-level label.

[0052] S2.4: Prune the candidate state transition graph containing abnormal transition edges into subgraphs, remove segments with continuous non-abnormal edges that do not meet the pruning threshold, and finally output a structured state transition graph.

[0053] S3: Based on the state transition graph, when a timestamp lag is found in the log modality corresponding to any transition path, the operation chain backtracking is activated, a virtual calibration node is inserted, and a tag is attached.

[0054] S3.1: Based on the state transition graph, extract the log modal timestamp sequence associated with each jump abnormal transition edge, align it with the timestamp field, calculate the cross-modal lag matrix, and identify lag transition paths with time lag exceeding the ratio threshold.

[0055] The field alignment process employs a synchronous time synchronization mechanism and addresses issues such as inconsistent time units and differences in recording granularity across different modalities. After alignment, the time interval between the source and target nodes in the log modality is calculated and combined with the corresponding time differences in other modalities (such as operation trajectory modality and video frame modality) to form a cross-modal lag matrix. Each element of the cross-modal lag matrix represents the degree of lag of the log modality relative to other modalities along a given jump path.

[0056] The ratio threshold is set as the maximum alignment offset of different signals / the time window span of the same behavior. In cross-modal data alignment research (such as multi-sensor data fusion, modal synchronization, video-speech alignment, etc.), a lag ratio of 0.2 to 0.3 is often considered to have caused significant information misalignment, and the ratio threshold is set accordingly.

[0057] S3.2: For lagging jump paths, recursively search backward from the starting node and extract the shortest path segment with consecutive abnormal jump edges as the backtracking sub-chain to avoid computational redundancy caused by full graph traversal.

[0058] The specific approach involves starting from the source node of a lagging jump edge and tracing back along the graph structure to all upstream nodes with connections, limiting the tracking to edges also marked as anomalous jumps. During the tracing process, each time an anomalous edge is encountered, the nodes at both ends are added to the candidate backtracking path set; if a non-anomalous edge is encountered or the path tracing depth limit is reached (e.g., no more than 5 hops), the current recursive branch terminates. This method effectively extracts a shortest path segment composed of consecutive anomalous jump edges, forming a local backtracking sub-chain. This backtracking sub-chain covers critical links where time lag cumulative effects may occur while avoiding the time and space complexity issues of a full graph search.

[0059] S3.3: In the backtracking subchain, determine the position of the weakest mode segment, insert a virtual calibration node with time interpolation markers, and attach the corresponding mode type label and delay estimate to the node.

[0060] In this embodiment of the invention, the location of the weakest modality segment is determined by weighted calculation based on indicators such as timestamp coverage density of modal data, number of synchronization events, consistency deviation, and data integrity, to quantitatively evaluate the time synchronization capability of different modalities on the same operation path. The calculation identifies the modal segment with the lowest synchronization confidence in the backtracking sub-chain, i.e., the operation segment most prone to lag or misalignment.

[0061] Insert a virtual calibration node at the corresponding paragraph. This virtual calibration node does not represent an actual operation but is a structural placeholder used to indicate possible synchronization breakpoints on the timeline. The virtual calibration node carries the following three types of attribute information: a time interpolation marker, indicating where the node should be pushed to compensate for synchronization delays; a modality type label, recording the mode in which the node operates; and a delay estimate, calculated based on the cross-modal lag matrix of the preceding and following nodes.

[0062] S3.4: Map the backtracking subchain structure after inserting the virtual calibration node back to and update the state transition graph to form the post-calibration state transition graph.

[0063] In this embodiment of the invention, the original jump edges need to be trimmed and replaced during the mapping process.

[0064] Specifically, for any original jump edge, if a virtual calibration node is inserted in the middle, it needs to be split into two new edges. The weights of these two new edges inherit a portion of the original edge's value, and edge weight compensation is performed based on the delay estimate. Simultaneously, to ensure the directionality and temporal consistency of the graph structure, an updated topological sort index needs to be maintained to assist in path planning and rhythm anomaly detection.

[0065] The calibrated state transition graph not only has the ability to correct structural deviations in hysteresis modes, but also enhances the graph model's ability to perceive and adjust multimodal time mismatches through the label information of virtual nodes.

[0066] S4: Map the calibrated state transition graph onto the preset process topology graph, perform subgraph sliding matching of length n, if n consecutive time series state nodes cannot form a connected subgraph in the process topology graph, it is determined to be a structural free segment and the start and end node indices are output as risk warning points, where n is a constant.

[0067] S4.1: Extract time-series state nodes arranged in chronological order from the calibrated state transition graph, and construct a set of sliding sub-paths consisting of n consecutive state nodes using a sliding window of fixed length n.

[0068] It should be noted that n is set such that the average time span of the sliding sub-path is not less than the global expected window, thus avoiding mismatch with the rhythm judgment result.

[0069] S4.2: For each sliding sub-path, sequentially search for paths in the flow structure diagram where the nodes are in the same order and any adjacent nodes are directly connected by directed edges in the flow structure diagram; if any node cannot be matched one-to-one with any node in the flow structure diagram during the mapping process, or there are no effective topological edges connecting adjacent nodes, then mark the corresponding sliding sub-path as a structurally disconnected segment.

[0070] For example, for each sliding sub-path, the following verification logic is executed sequentially in the preset process topology graph (which is a static directed graph representing the expected business or device behavior process):

[0071] Determine if each state node in the sliding sub-path has a unique corresponding node in the flow topology graph. If all nodes have a corresponding node, check in sequence whether adjacent nodes in the sliding path are directly connected by directed edges in the flow graph. If any node has no corresponding mapping in the flow graph, or if adjacent nodes have no topological connection edges in the flow graph, the sliding sub-path is marked as a structurally disconnected segment. Otherwise, the path is considered to be consistent with the flow graph structure and belongs to the expected behavior path.

[0072] The core of the above operations in this invention lies in stripping away non-process-oriented behaviors through structural consistency verification and constructing candidate regions for structural anomalies.

[0073] S4.3: When k consecutive sliding sub-paths are all determined to be structurally disconnected segments, and at least one of them contains a state node whose source is a calibration insertion node, the corresponding sub-path is marked as a structural free segment, and the corresponding start and end node indices are output as risk warning points.

[0074] To improve the robustness of anomaly detection and avoid oversensitivity to sporadic structural drift, joint constraints on continuity and calibration nodes are applied, as follows:

[0075] If k consecutive disconnected segments (e.g., k=3) appear, the consecutive segments are considered as candidate regions for structural anomalies. In the above consecutive disconnected segments, if at least one state node originates from the virtual calibration node inserted in stage S3, or any sub-path contains ≥1 rhythmic anomaly edge with a medium-high level of differentiation, then the anomaly is considered more likely to be caused by synchronous offset propagation due to modal mismatch. The set of sub-paths that meet the above two conditions is integrated into a structural free segment. The indices of the start and end nodes in this segment in the original state time series are output as risk warning points to indicate potential behavioral offset risks.

[0076] Through the above operations, segments that still exhibit structural derailment behavior after modal calibration can be effectively identified, especially sub-paths with continuous structural drift before and after the calibration node. These are often important manifestations of incomplete elimination of synchronization deviations in multimodal time-series fusion.

[0077] Furthermore, this embodiment also provides a dynamic identification system for internal control risks based on a multimodal model, such as... Figure 2 As shown, including,

[0078] The semantic extraction module 100 extracts a set of feature vectors for shared semantics from data of multiple modalities using a fusion attention mechanism, and performs serialization processing on the time dimension to obtain an internal control operation sequence tensor.

[0079] The state analysis module 200 performs unsupervised state transition analysis on the internal control operation sequence tensor, identifies rhythmic anomaly segments in multi-stage operations with intervals lower than the global expected window, and constructs a state transition graph.

[0080] The backtracking calibration module 300, based on the state transition graph, activates the operation chain backtracking, inserts a virtual calibration node and attaches a tag when it finds that there is a timestamp lag in the log modality corresponding to any transition path;

[0081] The structure detection module 400 maps the calibrated state transition diagram onto the preset process topology diagram and performs subgraph sliding matching of length n. If n consecutive time series state nodes cannot form a connected subgraph in the process topology diagram, it is determined to be a free structural segment and the start and end node indices are output as risk warning points, where n is a constant.

[0082] This embodiment also provides a computer device applicable to the dynamic identification method of internal control risks based on a multimodal model, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the dynamic identification method of internal control risks based on a multimodal model as proposed in the above embodiment.

[0083] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0084] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements the method for dynamic identification of internal control risks based on a multimodal model as proposed in the above embodiments.

[0085] In summary, this invention, through unsupervised state transition analysis, can dynamically identify hidden abnormal transition segments in behavioral patterns without relying on manually set rules, thus enhancing the adaptive capability of risk identification. When multimodal logs exhibit time lag, the operation chain backtracking and virtual calibration mechanisms ensure the temporal consistency and causal integrity of cross-modal behavioral chains, significantly reducing the probability of misjudgments due to inconsistencies in recording. The sliding matching method maps state sequences to the process topology graph, effectively identifying structurally mismatched fragments and accurately locating the start and end points of risky behaviors. In conclusion, this invention possesses high adaptability, high robustness, and high interpretability, providing strong technical support for the dynamic identification of internal control risks.

[0086] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for dynamic identification of internal control risks based on a multimodal model, characterized in that: Includes: The data from multiple modalities is processed using a fusion attention mechanism to extract a set of feature vectors for shared semantics across multiple modalities. This set of feature vectors is then input into a temporal awareness network for time-dimensional serialization processing to obtain an internal control operation sequence tensor. The data from multiple modalities includes approval logs, behavioral trajectories, operation timestamps, and video surveillance data. Unsupervised state transition analysis is performed on the internal control operation sequence tensor to identify rhythmic anomalies in multi-stage operations with intervals lower than the global expected window, and a state transition graph is constructed. Based on the state transition graph, when a timestamp lag is found in the log modality corresponding to any transition path, the operation chain backtracking is activated, a virtual calibration node is inserted and a tag is attached. The calibrated state transition graph is mapped onto the preset process topology graph. Subgraph sliding matching of length n is performed. If n consecutive time series state nodes cannot form a connected subgraph in the process topology graph, it is determined to be a structural free segment and the start and end node indices are output as risk warning points, where n is a constant.

2. The method for dynamic identification of internal control risks based on a multimodal model as described in claim 1, characterized in that: The extracted multimodal shared semantic feature vector set includes: Based on the timestamp field, a fixed-length sliding window strategy is used to align the original data of each modality, construct a sequence of synchronized data blocks, and remove segments with cross-modal time window overlap rates below a set threshold. The synchronous data block sequence is input into a cross-modal attention network to identify the dominant contribution of key modalities to behavioral features within the current time window at the semantic level, and a shared semantic tensor is generated accordingly to suppress weakly correlated modal noise.

3. The method for dynamic identification of internal control risks based on a multimodal model as described in claim 1, characterized in that: The unsupervised state transition analysis includes: The internal control operation sequence tensor is segmented into windows along the time dimension. Within each segment, the center of the operation state cluster is calculated using an adaptive clustering distance function to generate a set of time series state nodes. Based on a time-series set of state nodes, an unsupervised method is used to construct a candidate graph for state transitions, where each edge has a weight assigned according to the order of operations and the degree of behavioral variability between nodes.

4. The method for dynamic identification of internal control risks based on a multimodal model as described in claim 3, characterized in that: The construction of the state transition graph includes: Based on the global expected window length, rhythmic coherence constraint detection is performed on the state transition candidate graph, and any edge whose state transition interval is less than the global expected window is marked as an abnormal transition edge; The candidate state transition graph containing abnormal transition edges is pruned into subgraphs, and segments with continuous non-abnormal edges below the pruning threshold are removed, resulting in a structured state transition graph.

5. The method for dynamic identification of internal control risks based on a multimodal model as described in claim 1, characterized in that: The activation operation chain backtracking includes: Based on the state transition graph, the log modal timestamp sequence associated with each abnormal transition edge is extracted, and after being aligned with the timestamp field, the cross-modal lag matrix is ​​calculated to identify lag transition paths with a time lag higher than the ratio threshold. For delayed jump paths, recursively search backward from the starting node and extract the shortest path segment with consecutive abnormal jump edges as the backtracking sub-chain.

6. The method for dynamic identification of internal control risks based on a multimodal model as described in claim 5, characterized in that: The process of inserting the virtual calibration node and attaching the tag includes: In the backtracking subchain, the modal data indicators are used for weighted calculation. The modal segment with the lowest synchronization confidence in the backtracking subchain is identified by calculation. A virtual calibration node with time interpolation mark is inserted and the corresponding modal type label and delay estimate are attached to the node. The backtracking subchain structure after the insertion of the virtual calibration node is mapped back and the state transition graph is updated to form the post-calibration state transition graph.

7. The method for dynamic identification of internal control risks based on a multimodal model as described in claim 1, characterized in that: The process of mapping the calibrated state transition diagram to a preset flow topology diagram includes: Extract the time-series state nodes arranged in chronological order from the calibrated state transition graph, and construct a set of sliding sub-paths consisting of n consecutive state nodes using a sliding window of fixed length n. For each sliding sub-path, sequentially search the flow structure graph for paths where the nodes are in the same order and any adjacent nodes are directly connected by a directed edge in the flow structure graph.

8. A dynamic identification system for internal control risks based on a multimodal model, based on the dynamic identification method for internal control risks based on a multimodal model as described in any one of claims 1 to 7, characterized in that: Also includes: The semantic extraction module uses a fusion attention mechanism to extract a set of feature vectors for shared semantics from data across multiple modalities. This set of feature vectors is then input into a temporal awareness network for time-dimension serialization to obtain an internal control operation sequence tensor. The data from the multiple modalities includes approval logs, behavioral trajectories, operation timestamps, and video surveillance data. The state analysis module performs unsupervised state transition analysis on the internal control operation sequence tensor, identifies rhythmic anomaly segments in multi-stage operations with intervals lower than the global expected window, and constructs a state transition graph. The backtracking calibration module, based on the state transition graph, activates the operation chain backtracking when it finds that there is a timestamp lag in the log modality corresponding to any transition path, inserts a virtual calibration node and attaches a tag. The structure detection module maps the calibrated state transition graph onto a preset process topology graph and performs a subgraph sliding match of length n. If n consecutive time series state nodes cannot form a connected subgraph in the process topology graph, it is determined to be a structural free segment and the start and end node indices are output as risk warning points, where n is a constant.

9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the internal control risk dynamic identification method based on a multimodal model as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the dynamic identification method for internal control risks based on a multimodal model as described in any one of claims 1 to 7.

Images