An office abnormal behavior recognition method based on dynamic behavior graph

Abstract

The application provides an office abnormal behavior recognition method based on a dynamic behavior graph in the technical field of computer vision and intelligent monitoring, comprising: collecting a work station video stream and multi-modal sensing of employee identity, head, hand, line of sight, handheld object, body displacement, desktop object and hand-desktop interaction and other key behavior nodes, constructing a dynamically updated behavior state graph, containing node feature encoding, time sequence correlation edge and weight evolving over time; extracting individualized time sequence mode, interaction correlation and habit features based on historical graph sequences, forming a behavior fingerprint and establishing a dynamic baseline model; comparing the deviation degree of the current graph and the baseline, comprehensively analyzing abnormal behavior and triggering graded alarm. The application has the advantages that: the adaptability of abnormal recognition to individual behavior habits, the modeling ability of multi-node time sequence correlation logic, the discrimination accuracy of hand-object interaction semantics, and the dynamic modeling and generalization ability of individualized behavior baseline are greatly improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent monitoring technology, and in particular to a method for identifying abnormal office behavior based on dynamic behavior mapping. Background Technology

[0002] As enterprises increasingly demand higher operational efficiency, employee health and wellness, and intelligent building management, employee behavior perception and anomaly recognition in office environments have become a research hotspot in the fields of computer vision and intelligent monitoring. Traditional management methods, such as manual patrols and rule-based electronic attendance, are not only costly in terms of manpower but also struggle to handle complex, diverse, and subtle behavioral anomalies. While deep learning-based behavior recognition technology has developed rapidly in recent years, its direct application to the specific, semi-enclosed environment of an office still faces significant technical bottlenecks and challenges.

[0003] First, current mainstream behavior recognition methods generally employ a "general model" and "fixed threshold" judgment strategy, severely neglecting the inherent differences in behavioral habits among individuals. Office workers have highly individualized work styles; for example, some employees frequently get up to stimulate inspiration, while others prefer to remain hunched over for extended periods. Current anomaly detection algorithms, whether based on traditional machine learning or deep neural networks, mostly aim to learn a universal decision boundary applicable to all individuals. This "one-size-fits-all" approach directly leads to rigid anomaly judgment criteria. For individuals whose behavioral rhythms deviate from the statistical mean, it easily generates a large number of false positives (classifying normal individual behavior as abnormal) or false negatives (failing to identify the true anomalies of specific individuals), greatly limiting the system's practicality and reliability.

[0004] Secondly, existing technologies have limited perception dimensions and temporal modeling capabilities, often confined to classifying isolated actions in single-frame images or very short video clips. They severely lack the ability to model the temporal logical connections and dynamic evolutionary relationships between multiple behavioral nodes in an office setting. Abnormal office behavior often manifests as coordinated changes across multiple nodes, such as head posture, gaze direction, hand movements, and the state of desktop items, over time. Existing methods fail to fully explore the temporal dependencies and logical constraints between these nodes (e.g., the temporal correlation between hand movements and changes in desktop item states, and the coordinated rhythm of gaze direction and head posture changes). More critically, existing technologies completely ignore the inherent differences in temporal rhythms, action frequencies, and state transition patterns among individuals under normal working conditions, failing to use these individualized temporal pattern characteristics as a judgment criterion. This makes it difficult for the system to distinguish between accidental random fluctuations and potentially risky abnormal patterns based on the inherent logical chain of behavioral development, and it lacks the ability to accurately identify the causal chains of complex anomalies.

[0005] Furthermore, existing technologies suffer from insufficient modeling depth for highly discriminative interactive elements in office scenarios, particularly the states of desktop objects and their correlation with hand movements. In an office environment, employee interactions with desktop objects (such as phones, documents, water cups, and keyboards) are the core basis for determining their work status and intentions. However, current behavior recognition frameworks often focus on human posture or hand trajectories themselves, neglecting the objects affected by hand movements and their state changes (such as a phone being picked up or a document being flipped through), and failing to construct the semantic interaction relationship between "hand movements and desktop objects." This information gap prevents the system from effectively distinguishing between "normal work operations" and "non-work-related desktop operations," thus losing the ability to recognize such common and important anomalies.

[0006] Finally, while advanced models such as dynamic graph neural networks have been explored in the field of video anomaly detection, they have not yet been adapted for the highly personalized and interactive scenario of office workstations, which are centered on individual employees. Existing graph network-based methods mostly focus on modeling group behavior in public areas or industrial scenarios, and their node definition and relationship reasoning mechanisms are difficult to directly transfer to workstation monitoring scenarios. Specifically, there is a lack of a dynamic graph structure that can uniformly represent multi-source heterogeneous information such as employee identity, head posture, gaze focus, hand movements, held objects, body displacement, and the state of items on the desktop. Furthermore, it has failed to establish a personalized behavioral baseline model for specific employees that is dynamically evolving and incorporates individual temporal habits.

[0007] In summary, existing technologies are limited by rigid judgment criteria, a lack of modeling for the temporal logical relationships and dynamic evolution of nodes, insufficient perception of hand-object interaction semantics, and a lack of dynamic graph structures adapted to workstation scenarios. These limitations make it difficult to meet the urgent need for personalized, structured, and refined anomaly identification of employee behavior in an office environment. Therefore, providing a method for identifying abnormal office behavior based on dynamic behavior graphs, which improves the adaptability of anomaly identification to individual behavioral habits, the ability to model the temporal logical relationships of multiple nodes, the accuracy of hand-object interaction semantics discrimination, and the dynamic modeling and generalization ability of personalized behavioral baselines, has become a pressing technical problem to be solved. Summary of the Invention

[0008] The technical problem to be solved by this invention is to provide an office abnormal behavior identification method based on dynamic behavior graph, which improves the adaptability of abnormal identification to individual behavioral habits, the ability to model multi-node temporal correlation logic, the discrimination accuracy of hand-object interaction semantics, and the dynamic modeling and generalization ability of personalized behavioral baseline.

[0009] This invention provides a method for identifying abnormal office behavior based on dynamic behavior graphs, comprising the following steps: Step S1: Acquire video streams from the office workstation area and perform multimodal perception processing on the video frame sequence of the video stream to extract the state sequences of multiple key behavioral nodes; the key behavioral nodes include employee identity nodes, head posture nodes, hand action nodes, gaze direction nodes, handheld object state nodes, body displacement trajectory nodes, desktop item state nodes, and hand-desktop interaction state nodes. Step S2: Construct a dynamically updated behavior state graph for each employee; the behavior state graph uses key behavior nodes as the node set, edges reflecting the temporal correlation between nodes as the edge set, and includes time dimension information; the construction of the behavior state graph includes encoding node features, establishing temporal correlation edges according to behavior logic, and dynamically updating node features and edge weights as the video stream is input. Step S3: Based on the employee's historical behavior state map sequence, extract the employee's unique behavioral fingerprint and establish a personalized dynamic behavior baseline model; the behavioral fingerprint includes temporal pattern features, interaction association features and individual habit features; Step S4: By comparing the degree of deviation between the current behavior state map and the dynamic behavior baseline model, a comprehensive judgment is made on the abnormal behavior in the office to trigger a graded alarm.

[0010] Furthermore, in step S1, the employee identity node is determined through face recognition and personnel re-identification technology; the head posture node extracts pitch angle, yaw angle, and roll angle parameters through a head posture estimation algorithm; the hand action node extracts hand position, action category, and movement trajectory based on hand key point detection and gesture recognition; the gaze direction node determines the gaze focus area through gaze estimation technology; the handheld object status node identifies the category of the item held by the employee through target detection; the body displacement trajectory node extracts body displacement trajectory and posture changes through human skeletal key point tracking; the desktop item status node obtains the position, category, and status changes of the item by performing target detection and status recognition on the desktop area; and the hand-desktop interaction status node determines the interaction relationship between the hand and the desktop item through spatial relationship reasoning.

[0011] Furthermore, in step S2, the encoding of the node features further includes: encoding the head pose node as a three-dimensional angle vector, encoding the hand action node as a combination vector of action category and position coordinates, and encoding the desktop item status node as a combination vector of item category, position, and status change flags.

[0012] Furthermore, in step S2, the establishment of the temporal association edge further includes: establishing an "action-object" association edge between the hand action node and the desktop item state node; establishing a "line of sight-head coordination" association edge between the line of sight direction node and the head posture node; establishing a "movement-holding" association edge between the body displacement trajectory node and the handheld object state node; and dynamically calculating the weight of each association edge based on the temporal proximity and logical relevance of the node state changes.

[0013] Furthermore, in step S2, the edge weights are dynamically calculated using a graph attention mechanism.

[0014] Furthermore, in step S3, the temporal pattern features include the duration, frequency, rhythm, and transition probability of the state sequence of each key behavioral node; the interaction association features include the association strength and collaboration pattern between multiple nodes; and the individual habit features are established through long-term observation of the typical state distribution, temporal transition rules, and normal interaction patterns of multiple nodes of the employee under normal working conditions.

[0015] Furthermore, in step S4, the comprehensive judgment further includes: using a graph neural network to embed the current behavior state map, calculating its structural similarity and feature deviation from the dynamic behavior baseline model, and marking time periods with deviations exceeding an adaptive threshold as potential anomalies.

[0016] Furthermore, in step S4, the abnormal behavior includes at least one of the following: prolonged inattention, frequent use of mobile phones, non-work-related desktop operations, prolonged absence from work, and abnormal body posture.

[0017] Furthermore, the criteria for identifying prolonged inattention include at least one of the following: abnormally high frequency of gaze wandering, deviation of gaze duration distribution from the baseline, and decreased head posture stability; the criteria for identifying frequent mobile phone use include at least one of the following: frequent switching of the handheld object to the mobile phone, abnormal correlation between gaze direction and mobile phone position, and hand movement rhythm inconsistent with normal usage patterns; the criteria for identifying non-work-related desktop operations include hand-to-desktop interaction patterns deviating from normal work practices; the criteria for identifying prolonged absence from the work station include body displacement trajectories exceeding the work station area and duration exceeding the individual baseline threshold; and the criteria for identifying abnormal body posture include body posture duration or frequency of change deviating from the normal range.

[0018] Furthermore, step S4 also includes: Based on the changes in behavior patterns within a sliding window of the most recent N working days corresponding to the dynamically updated dynamic behavior baseline model of the employee, the adaptive threshold is dynamically adjusted to adapt to the behavioral differences of the same employee at different working hours and different task stages, so as to maintain the timeliness and adaptability of the dynamic behavior baseline model.

[0019] The advantages of this invention are: 1. By constructing eight types of multimodal perception nodes covering employee identity, head posture, gaze direction, hand movements, held object status, body displacement trajectory, desktop object status, and hand-desk interaction status, a unique behavioral fingerprint containing temporal patterns, interaction associations, and individual habit characteristics is extracted for each employee. A personalized behavioral baseline that dynamically evolves with the video stream is established, and the judgment threshold is adaptively adjusted using a sliding window. This ensures that the anomaly detection criteria closely follow the individual's behavioral rhythm changes at different times and task stages, fundamentally improving adaptability to individual habits. Furthermore, association edges reflecting temporal proximity and logical relevance are established between nodes. The edge weights are dynamically calculated using a graph attention mechanism, enabling the dynamic behavioral graph to accurately depict the interconnected evolution and internal causal chains of multiple nodes along the timeline, effectively compensating for isolated nodes. The limitations of motion analysis are addressed. Simultaneously, object detection and spatial relationship reasoning are used to specifically perceive the type of handheld object, changes in the state of desktop items, and the actual interaction between the hand and desktop items. This serves as a key criterion for distinguishing normal office operations from non-work-related behaviors, significantly improving the accuracy of interaction semantic discrimination. Finally, a graph neural network is used to compare the structural similarity and feature deviation between the current graph and the individual baseline. Combined with sliding window data, the anomaly detection threshold is dynamically calibrated, enabling the baseline model to both generalize and update with the natural drift of employees' long-term behavioral habits and maintain timeliness in response to short-term task changes. This systematically achieves precise adaptation to individual behavioral habits, structured modeling of multi-node temporal correlation logic, refined discrimination of hand-object interaction semantics, and dynamic evolution and generalization of personalized behavioral baselines.

[0020] 2. Based on behavioral fingerprints, a unique and dynamically updated behavioral baseline model is established for each employee. This baseline model is not static, but rather, through long-term observation, it deeply mines and encodes the temporal pattern characteristics, interaction correlation characteristics, and individual habit characteristics of specific employees. This design makes anomaly detection no longer rely on rigid general thresholds, but uses the employee's own "normal state" as a reference. For example, for employees who are accustomed to getting up frequently, their threshold for judging "long-term absence from work" will be adaptively higher than that of employees who are hunched over their desks, thereby accurately distinguishing between genuine absence anomalies and personalized normal behavioral rhythms, significantly improving the robustness and reliability of the system in real and complex office environments.

[0021] 3. By constructing a dynamic graph with eight key behavioral nodes as the node set and edges reflecting temporal relationships as the edge set, behavioral modeling is elevated from "single-point classification" to the level of "logical chain analysis." In particular, semantically specific "action-object" association edges are established between different nodes (such as "hand-desktop object"), and edge weights are dynamically calculated based on the temporal proximity and logical relevance of node state changes. This means that the system can not only identify "hand moving" and "phone being picked up," but also determine the temporal sequence and logical causal relationship between these two events. This precise capture of the inherent logic of behavioral development enables the system to effectively distinguish between accidental random fluctuations and potentially risky abnormal patterns based on the dynamic evolution chain, greatly enhancing the ability to identify complex abnormal causal chains.

[0022] 4. In addition to identifying the categories and states of desktop objects and handheld items through object detection, the system further establishes a dedicated "hand-desktop interaction state node" and uses spatial relationship reasoning to determine the interaction relationship between the hand and desktop objects. The encoding of node features quantifies this interaction semantics. For example, "desktop object state" is encoded as a combined vector containing "state change flags." This design enables the system to have a sophisticated cognitive ability to understand "what the hand is operating" and "whether the operation causes a change in the state of the object." As a result, the system can distinguish "normal work operations" (such as typing on a keyboard or flipping through documents) from "non-work-related desktop operations" (such as playing with a mobile phone for a long time) with high discrimination accuracy, directly addressing the lack of existing technologies in identifying such common and important anomalies.

[0023] 5. First, a dynamic graph structure centered on the employee's own behavioral nodes and oriented towards single-employee scenarios is defined. Second, a graph attention mechanism is used to dynamically calculate edge weights, which enables the graph to adaptively focus on the most discriminative node associations under the current behavioral pattern, rather than relying on fixed relation weights. Furthermore, a graph neural network is used to embed the current graph and calculate its structural similarity and feature deviation with the baseline model. At the same time, a sliding window mechanism is introduced to dynamically adjust the judgment threshold. This entire mechanism ensures that the system can not only uniformly represent heterogeneous information such as posture, action, and object in the spatial dimension, but also allows the baseline model to smoothly evolve with changes in the employee's recent behavioral patterns in the temporal dimension, maintaining the model's timeliness and adaptability. Attached Figure Description

[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0025] Figure 1 This is a flowchart of an office abnormal behavior identification method based on dynamic behavior graphs according to the present invention. Detailed Implementation

[0026] The technical solution in this application embodiment has the following general idea: Using multimodal perception technology, the state sequences of eight key behavioral nodes—employee identity, head posture, gaze direction, hand movements, held objects, body displacement, desktop items, and hand-desktop interactions—are extracted from the workstation video stream. These nodes for each employee, along with their temporal association edges established according to behavioral logic, are then used to construct a dynamically updated behavioral state graph. Node features are vectorized and encoded, while edge weights are dynamically calculated using a graph attention mechanism to accurately characterize the temporal proximity and logical causal relationships between behaviors. Based on this, unique temporal patterns, interaction associations, and habitual features are extracted from the historical graph sequence observed over a long period, forming a personalized behavioral fingerprint and establishing a system based on this fingerprint. The dynamically evolving behavioral baseline model ensures that anomaly detection is based entirely on the employee's "normal state" rather than relying on rigid universal thresholds. Finally, a graph neural network is used to quantitatively compare the structural similarity and feature deviation between the current behavioral map and the baseline model. A sliding window mechanism is introduced to adaptively adjust the judgment threshold based on recent behavioral pattern changes, thereby enabling graded alerts for various abnormal behaviors such as inattention, frequent mobile phone use, and absence from work. This closed-loop process, from multi-source perception, structured map construction, individual baseline modeling to adaptive judgment, systematically solves the core shortcomings of existing technologies in terms of individual habit adaptability, multi-node temporal correlation logic modeling, hand-object interaction semantic discrimination, and dynamic baseline generalization.

[0027] Please refer to Figure 1 As shown, a preferred embodiment of the present invention, a method for identifying abnormal office behavior based on dynamic behavior graphs, includes the following steps: Step S1: Acquire video streams from the office workstation area and perform multimodal perception processing on the video frame sequence of the video stream to extract the state sequences of multiple key behavioral nodes; the key behavioral nodes include employee identity nodes, head posture nodes, hand action nodes, gaze direction nodes, handheld object state nodes, body displacement trajectory nodes, desktop item state nodes, and hand-desktop interaction state nodes. In practice, video streams are captured by cameras deployed in the office workstation area. Facial recognition technology is used to create a unique user ID for each employee, and their behavior within the workstation area is continuously tracked. Subsequently, object detection and pose estimation algorithms are used to continuously process the video frame sequence and extract the state sequences of key behavioral nodes.

[0028] Step S2: Construct a dynamically updated behavior state graph for each employee; the behavior state graph uses key behavior nodes as the node set, edges reflecting the temporal correlation between nodes as the edge set, and includes time dimension information; the construction of the behavior state graph includes encoding node features, establishing temporal correlation edges according to behavior logic, and dynamically updating node features and edge weights as the video stream is input. The behavioral state graph is represented as G=(V,E,T), where V is the set of key behavioral nodes, E is the set of associated edges between nodes, and T is the time dimension.

[0029] The update of the behavioral state graph follows a sliding time window mechanism and a feature decay strategy. Specifically, the system maintains a sliding time window of length L (e.g., L = 300 frames, corresponding to a duration of 10 seconds). The node feature encoding in the graph is no longer a snapshot of a single moment, but rather a feature set of all states of that node within the window, including state value, duration, and state transition time. When a new video frame arrives, the system pushes the latest decoded node state into the window and removes the state with the earliest timestamp. In the dynamic update of edge weights, the weight calculation of associated edges depends not only on the logical correlation between the states of two nodes at the current moment, but also on the temporal proximity decay factor. That is, the shorter the time interval between the occurrence of two node state change events, the higher their contribution to the current edge weight, while the contribution of events with a time interval exceeding the window length decays to zero. The temporal dimension information T is encoded as a relative temporal position embedding, which is concatenated with the node features and input into the graph attention layer. This allows the graph to perceive the order and temporal distance of different node states when calculating edge weights, thereby accurately depicting the temporal logic of the behavioral chain.

[0030] Step S3: Based on the employee's historical behavior state map sequence, extract the employee's unique behavioral fingerprint and establish a personalized dynamic behavior baseline model; the behavioral fingerprint includes temporal pattern features, interaction association features and individual habit features; Step S4: By comparing the degree of deviation between the current behavior state map and the dynamic behavior baseline model, a comprehensive judgment is made on the abnormal behavior in the office to trigger a graded alarm.

[0031] In step S1, the employee identity node is determined through facial recognition and personnel re-identification technology to identify the employee's identity at the current workstation; the head posture node extracts pitch, yaw, and roll angle parameters through a head posture estimation algorithm to characterize head orientation; the hand action node extracts hand position, action type (e.g., raising, lowering, clicking, grasping, etc.) and movement trajectory based on hand key point detection and gesture recognition; the gaze direction node determines the gaze focus area through gaze estimation technology to determine whether the gaze is focused on the screen, document, mobile phone, or other areas; the handheld object status node identifies the type of item held by the employee (e.g., mobile phone, pen, cup, etc.) through target detection; the body displacement trajectory node extracts body displacement trajectory and posture changes (e.g., sitting, leaning, leaning back, standing, etc.) through human skeletal key point tracking; the desktop item status node obtains the position, type, and status changes of items (e.g., mobile phone being picked up from the desktop, document being opened, etc.) through target detection and status recognition of the desktop area; and the hand-desktop interaction status node determines the interaction relationship between the hand and desktop items (e.g., contact, picking up, moving, etc.) through spatial relationship reasoning.

[0032] In a preferred implementation, multimodal perception processing does not perform the highest-precision calculation of all nodes for every video frame. Instead, it employs a heterogeneous perception pipeline architecture. Specifically, for computationally intensive perception tasks (such as hand keypoint detection and human skeleton tracking), processing is performed in real-time at the original frame rate of the video stream (e.g., 30fps). For computationally intensive perception tasks (such as gaze direction estimation and fine-grained recognition of desktop object states), asynchronous processing is performed at a lower frequency (e.g., 5fps or a keyframe extraction strategy). The node state sequences generated at different frequencies are synchronized to a unified timeline through a timestamp alignment module, forming a time-aligned multimodal state sequence. Simultaneously, the system includes a perception confidence evaluation mechanism: when the confidence of a node (e.g., gaze direction) falls below a preset threshold across multiple consecutive frames, the system initiates state inference and smooth interpolation based on the node's historical state sequence and its temporal coordination patterns with other strongly correlated nodes (e.g., head pose) to generate the optimal estimated state of the node at the current moment, thereby ensuring the continuity and integrity of the behavioral state map in the temporal dimension.

[0033] In step S2, the encoding of the node features further includes: encoding the head pose node as a three-dimensional angle vector, encoding the hand action node as a combination vector of action category and position coordinates, and encoding the desktop item status node as a combination vector of item category, position, and status change flags.

[0034] In step S2, the establishment of the temporal association edge further includes: establishing an "action-object" association edge between the hand action node and the desktop item state node; establishing a "line of sight-head coordination" association edge between the line of sight direction node and the head posture node; establishing a "movement-holding" association edge between the body displacement trajectory node and the handheld object state node; and dynamically calculating the weight of each association edge based on the temporal proximity and logical relevance of the node state changes.

[0035] In step S2, the edge weights are dynamically calculated using a graph attention mechanism.

[0036] In step S3, the temporal pattern features include the duration, frequency, rhythm, and transition probability of the state sequence of each key behavioral node (e.g., the distribution of the transition path and dwell time of the gaze between the screen, document, and mobile phone; the time interval between raising the hand and picking up or putting down items on the desktop; the trend and duration of the change in body posture from sitting to leaning or reclining); the interaction association features include the association strength and coordination pattern between multiple nodes (e.g., the coordination frequency of hand movements and changes in the state of items on the desktop; the degree of consistency between the gaze direction and head posture; the association pattern between the state of the held object and the body displacement trajectory); and the individual habit features are the typical state distribution, temporal transition rules, and normal interaction patterns of each behavioral node of the employee under normal working conditions, established through long-term observation.

[0037] In one specific implementation, behavioral fingerprint extraction and baseline model construction employ a strategy combining hierarchical statistics and graph embedding coding. First, the system analyzes all historical graph sequences of a single employee on a daily basis.

[0038] 1. Temporal pattern feature extraction: Statistically analyze the duration distribution of each node state within each workday (such as the total duration of eye focus on the screen and the histogram of single duration), state transition probability matrix (such as the first and second order transition probabilities of eye focus between "screen-phone-document"), and periodic rhythm features (such as the hourly frequency curve of hand lifting action).

[0039] 2. Interactive correlation feature extraction: Calculate the mutual information coefficient and temporal causal lag correlation of state changes between nodes, quantify the coordination strength between "head posture deviation" and "gaze direction switching", and the average delay time between "hand action start" and "desktop item state change".

[0040] 3. Construction of the Dynamic Behavioral Baseline Model: The extracted multidimensional statistics are normalized and encoded to form a fixed-length behavioral fingerprint vector for the employee. This fingerprint vector serves as the static anchor point of the baseline model. Simultaneously, the system uses a Gaussian Mixture Model (GMM) to fit the probability density of the employee's graph neural network embedding vectors over the past N working days, constructing the dynamic probability distribution part of the baseline model. The mean vector of the GMM represents the employee's most common graph structure mode, and the covariance matrix characterizes the acceptable fluctuation range of their daily behavior. Finally, this GMM model, together with the fixed fingerprint vector, constitutes the employee's complete personalized baseline model at time t.

[0041] In step S4, the comprehensive judgment further includes: using a graph neural network to embed the current behavior state map, calculating its structural similarity and feature deviation from the dynamic behavior baseline model, and marking time periods with deviations exceeding an adaptive threshold as potential anomalies.

[0042] Specifically, the comprehensive judgment is completed collaboratively by three modules: encoder, comparator, and decision-maker. First, the encoder module uses a Temporal Graph Attention Network (TGAT) as its backbone. The TGAT takes a continuous behavioral state graph (G_{tL},...,G_t}) within the current sliding time window as input, and after multi-layer graph convolution and temporal convolution operations, outputs a fixed-dimensional current behavior pattern embedding vector. This vector comprehensively represents the employee's behavioral state and the dynamic relationships between nodes within the current short-term window. Second, the comparator module calculates the probability density value of this current embedding vector and the GMM in the baseline model. The negative logarithm of this probability density is the feature deviation. Simultaneously, it calculates the Frobenius norm difference between the adjacency matrix of the current graph and the average adjacency matrix in the baseline model as a quantitative indicator of structural similarity. Finally, the feature deviation and structural similarity are weighted and summed to obtain a comprehensive anomaly score. When this comprehensive score exceeds an adaptive threshold, the decision-maker marks that time period as a potential anomaly and triggers different levels of alarms based on the score.

[0043] In step S4, the abnormal behavior includes at least one of the following: prolonged inattention, frequent use of mobile phones, non-work-related desktop operations, prolonged absence from work, and abnormal body posture.

[0044] The criteria for identifying prolonged inattention include at least one of the following: abnormally high frequency of gaze wandering, deviation of gaze duration from the baseline, and decreased head posture stability; the criteria for identifying frequent mobile phone use include at least one of the following: frequent switching of the handheld object to the mobile phone, abnormal duration of the association between the gaze direction and the mobile phone position, and hand movement rhythm inconsistent with the normal usage pattern; the criteria for identifying non-work-related desktop operations include hand-to-desktop interaction patterns that deviate from normal work practices; the criteria for identifying prolonged absence from the work station include body displacement trajectories exceeding the work station area and durations exceeding the individual baseline threshold; and the criteria for identifying abnormal body posture include body posture duration or frequency of change deviating from the normal range.

[0045] Step S4 further includes: Based on the changes in behavior patterns within a sliding window of the most recent N working days corresponding to the dynamically updated dynamic behavior baseline model of the employee, the adaptive threshold is dynamically adjusted to adapt to the behavioral differences of the same employee at different working hours and different task stages, so as to maintain the timeliness and adaptability of the dynamic behavior baseline model.

[0046] In one specific implementation, the dynamic adjustment of the adaptive threshold employs a sliding window calibration method based on quantile statistics. The system maintains a historical comprehensive anomaly score sequence of length M (e.g., M=30 days) for the employee. After the workday ends, the system calculates the comprehensive anomaly score for all time periods of the day and takes its 95th percentile as the daily threshold. The adaptive threshold is then set as the moving weighted average of the most recent M daily thresholds, with higher weights assigned to daily thresholds closer to the current date. Through this mechanism, if an employee's screen time has generally increased recently due to taking on new projects (a normal change in task phase), their daily threshold will rise accordingly, and the system will not misjudge this normal behavioral pattern shift as abnormal. Conversely, if an employee's behavioral pattern suddenly reverts to a historically low-activity state, but the comprehensive anomaly score is higher than the updated, higher threshold, the system can more sensitively detect genuine abnormal deviations.

[0047] In summary, the advantages of this invention are as follows: 1. By constructing eight types of multimodal perception nodes covering employee identity, head posture, gaze direction, hand movements, held object status, body displacement trajectory, desktop object status, and hand-desk interaction status, a unique behavioral fingerprint containing temporal patterns, interaction associations, and individual habit characteristics is extracted for each employee. A personalized behavioral baseline that dynamically evolves with the video stream is established, and the judgment threshold is adaptively adjusted using a sliding window. This ensures that the anomaly detection criteria closely follow the individual's behavioral rhythm changes at different times and task stages, fundamentally improving adaptability to individual habits. Furthermore, association edges reflecting temporal proximity and logical relevance are established between nodes. The edge weights are dynamically calculated using a graph attention mechanism, enabling the dynamic behavioral graph to accurately depict the interconnected evolution and internal causal chains of multiple nodes along the timeline, effectively compensating for isolated nodes. The limitations of motion analysis are addressed. Simultaneously, object detection and spatial relationship reasoning are used to specifically perceive the type of handheld object, changes in the state of desktop items, and the actual interaction between the hand and desktop items. This serves as a key criterion for distinguishing normal office operations from non-work-related behaviors, significantly improving the accuracy of interaction semantic discrimination. Finally, a graph neural network is used to compare the structural similarity and feature deviation between the current graph and the individual baseline. Combined with sliding window data, the anomaly detection threshold is dynamically calibrated, enabling the baseline model to both generalize and update with the natural drift of employees' long-term behavioral habits and maintain timeliness in response to short-term task changes. This systematically achieves precise adaptation to individual behavioral habits, structured modeling of multi-node temporal correlation logic, refined discrimination of hand-object interaction semantics, and dynamic evolution and generalization of personalized behavioral baselines.

[0048] 2. Based on behavioral fingerprints, a unique and dynamically updated behavioral baseline model is established for each employee. This baseline model is not static, but rather, through long-term observation, it deeply mines and encodes the temporal pattern characteristics, interaction correlation characteristics, and individual habit characteristics of specific employees. This design makes anomaly detection no longer rely on rigid general thresholds, but uses the employee's own "normal state" as a reference. For example, for employees who are accustomed to getting up frequently, their threshold for judging "long-term absence from work" will be adaptively higher than that of employees who are hunched over their desks, thereby accurately distinguishing between genuine absence anomalies and personalized normal behavioral rhythms, significantly improving the robustness and reliability of the system in real and complex office environments.

[0049] 3. By constructing a dynamic graph with eight key behavioral nodes as the node set and edges reflecting temporal relationships as the edge set, behavioral modeling is elevated from "single-point classification" to the level of "logical chain analysis." In particular, semantically specific "action-object" association edges are established between different nodes (such as "hand-desktop object"), and edge weights are dynamically calculated based on the temporal proximity and logical relevance of node state changes. This means that the system can not only identify "hand moving" and "phone being picked up," but also determine the temporal sequence and logical causal relationship between these two events. This precise capture of the inherent logic of behavioral development enables the system to effectively distinguish between accidental random fluctuations and potentially risky abnormal patterns based on the dynamic evolution chain, greatly enhancing the ability to identify complex abnormal causal chains.

[0050] 4. In addition to identifying the categories and states of desktop objects and handheld items through object detection, the system further establishes a dedicated "hand-desktop interaction state node" and uses spatial relationship reasoning to determine the interaction relationship between the hand and desktop objects. The encoding of node features quantifies this interaction semantics. For example, "desktop object state" is encoded as a combined vector containing "state change flags." This design enables the system to have a sophisticated cognitive ability to understand "what the hand is operating" and "whether the operation causes a change in the state of the object." As a result, the system can distinguish "normal work operations" (such as typing on a keyboard or flipping through documents) from "non-work-related desktop operations" (such as playing with a mobile phone for a long time) with high discrimination accuracy, directly addressing the lack of existing technologies in identifying such common and important anomalies.

[0051] 5. First, a dynamic graph structure centered on the employee's own behavioral nodes and oriented towards single-employee scenarios is defined. Second, a graph attention mechanism is used to dynamically calculate edge weights, which enables the graph to adaptively focus on the most discriminative node associations under the current behavioral pattern, rather than relying on fixed relation weights. Furthermore, a graph neural network is used to embed the current graph and calculate its structural similarity and feature deviation with the baseline model. At the same time, a sliding window mechanism is introduced to dynamically adjust the judgment threshold. This entire mechanism ensures that the system can not only uniformly represent heterogeneous information such as posture, action, and object in the spatial dimension, but also allows the baseline model to smoothly evolve with changes in the employee's recent behavioral patterns in the temporal dimension, maintaining the model's timeliness and adaptability.

[0052] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for identifying abnormal office behavior based on dynamic behavior graphs, characterized in that: Includes the following steps: Step S1: Acquire video streams from the office workstation area and perform multimodal perception processing on the video frame sequence of the video stream to extract the state sequences of multiple key behavioral nodes; the key behavioral nodes include employee identity nodes, head posture nodes, hand action nodes, gaze direction nodes, handheld object state nodes, body displacement trajectory nodes, desktop item state nodes, and hand-desktop interaction state nodes. Step S2: Construct a dynamically updated behavior state graph for each employee; the behavior state graph uses key behavior nodes as the node set, edges reflecting the temporal correlation between nodes as the edge set, and includes time dimension information; the construction of the behavior state graph includes encoding node features, establishing temporal correlation edges according to behavior logic, and dynamically updating node features and edge weights as the video stream is input. Step S3: Based on the employee's historical behavior state map sequence, extract the employee's unique behavioral fingerprint and establish a personalized dynamic behavior baseline model; the behavioral fingerprint includes temporal pattern features, interaction association features and individual habit features; Step S4: By comparing the degree of deviation between the current behavior state map and the dynamic behavior baseline model, a comprehensive judgment is made on the abnormal behavior in the office to trigger a graded alarm.

2. The method for identifying abnormal office behavior based on dynamic behavior graphs as described in claim 1, characterized in that: In step S1, the employee identity node is determined through face recognition and personnel re-identification technology; the head posture node extracts pitch, yaw, and roll angle parameters through a head posture estimation algorithm; the hand movement node extracts hand position, movement category, and motion trajectory based on hand key point detection and gesture recognition; the gaze direction node determines the gaze focus area through gaze estimation technology; the handheld object status node identifies the category of the item held by the employee through target detection; the body displacement trajectory node extracts body displacement trajectory and posture changes through human skeletal key point tracking; the desktop item status node obtains the position, category, and status changes of the item by performing target detection and status recognition on the desktop area; and the hand-desktop interaction status node determines the interaction relationship between the hand and the desktop item through spatial relationship reasoning.

3. The method for identifying abnormal office behavior based on dynamic behavior graphs as described in claim 1, characterized in that: In step S2, the encoding of the node features further includes: encoding the head pose node as a three-dimensional angle vector, encoding the hand action node as a combination vector of action category and position coordinates, and encoding the desktop item status node as a combination vector of item category, position, and status change flags.

4. The method for identifying abnormal office behavior based on dynamic behavior graphs as described in claim 1, characterized in that: In step S2, the establishment of the temporal association edge further includes: establishing an "action-object" association edge between the hand action node and the desktop item state node; establishing a "line of sight-head coordination" association edge between the line of sight direction node and the head posture node; establishing a "movement-holding" association edge between the body displacement trajectory node and the handheld object state node; and dynamically calculating the weight of each association edge based on the temporal proximity and logical relevance of the node state changes.

5. The method for identifying abnormal office behavior based on dynamic behavior graphs as described in claim 1, characterized in that: In step S2, the edge weights are dynamically calculated using a graph attention mechanism.

6. The method for identifying abnormal office behavior based on dynamic behavior graphs as described in claim 1, characterized in that: In step S3, the temporal pattern features include the duration, frequency, rhythm, and transition probability of the state sequence of each key behavioral node; the interaction association features include the association strength and collaboration pattern between multiple nodes; and the individual habit features are typical state distributions, temporal transition patterns, and normal interaction patterns of multiple nodes of the employee under normal working conditions, established through long-term observation.

7. The method for identifying abnormal office behavior based on dynamic behavior graphs as described in claim 1, characterized in that: In step S4, the comprehensive judgment further includes: using a graph neural network to embed the current behavior state map, calculating its structural similarity and feature deviation from the dynamic behavior baseline model, and marking time periods with deviations exceeding an adaptive threshold as potential anomalies.

8. The method for identifying abnormal office behavior based on dynamic behavior graphs as described in claim 1, characterized in that: In step S4, the abnormal behavior includes at least one of the following: prolonged inattention, frequent use of mobile phones, non-work-related desktop operations, prolonged absence from work, and abnormal body posture.

9. The method for identifying abnormal office behavior based on dynamic behavior graphs as described in claim 8, characterized in that: The criteria for identifying prolonged inattention include at least one of the following: abnormally high frequency of gaze wandering, deviation of gaze duration from the baseline, and decreased head posture stability; the criteria for identifying frequent mobile phone use include at least one of the following: frequent switching of the handheld object to the mobile phone, abnormal duration of the association between the gaze direction and the mobile phone position, and hand movement rhythm inconsistent with the normal usage pattern; the criteria for identifying non-work-related desktop operations include hand-to-desktop interaction patterns that deviate from normal work practices; the criteria for identifying prolonged absence from the work station include body displacement trajectories exceeding the work station area and durations exceeding the individual baseline threshold; and the criteria for identifying abnormal body posture include body posture duration or frequency of change deviating from the normal range.

10. The method for identifying abnormal office behavior based on dynamic behavior graphs as described in claim 7, characterized in that: Step S4 further includes: Based on the changes in behavior patterns within a sliding window of the most recent N working days corresponding to the dynamically updated dynamic behavior baseline model of the employee, the adaptive threshold is dynamically adjusted to adapt to the behavioral differences of the same employee at different working hours and different task stages, so as to maintain the timeliness and adaptability of the dynamic behavior baseline model.

Images