Supervisor on-site positioning and behavior compliance analysis method and system based on UWB and IMU fusion

Abstract

The invention discloses a supervision personnel on-site positioning and behavior compliance analysis method and system based on UWB and IMU fusion, and relates to the technical field of engineering management and intelligent sensing, and the method comprises the steps: collecting positioning, movement, myoelectricity and touch data through a multi-mode wearable device, fusing the UWB and IMU data through an error state Kalman filtering tight coupling algorithm, and carrying out the recognition of the UWB and IMU data through the UWB and IMU fusion. A filter noise covariance matrix is dynamically adjusted in combination with a lightweight NLOS recognition network, robust positioning is achieved, spatial postures, hand motion units and head-sight line features are extracted to construct micro-action vectors, and a TP-LSTM-CRF model is input to generate a time-position-action semantic behavior flow; a standard operation process digital twinborn model is established, behavior flow detection position omission and action violation are dynamically matched, operation quality is deduced in combination with action parameters, and finally results are integrated to generate a visual report containing a positioning track, a compliance rate and rectification suggestions.

Description

Technical Field

[0001] This invention relates to the fields of engineering management and intelligent sensing technology, and more specifically, to a method and system for on-site positioning and behavioral compliance analysis of supervisors based on the fusion of UWB and IMU. Background Technology

[0002] In the process of deepening digitalization in engineering construction, supervision, as a key hub for quality and safety control, has become a core requirement for breaking through traditional control models in terms of the traceability of on-site operations and the verification of compliance of behavior. Currently, large-scale industrial and infrastructure projects are characterized by heterogeneous operation scenarios, dense process connections, and refined control of key processes. The traditional supervision model, which relies on manual on-site recording and post-event accounting verification, can no longer meet the underlying needs of intelligent construction for process perception, risk prediction, and responsibility traceability. The authenticity of the supervisor's presence, the standardization of operation, and the depth of quality verification at key processes directly determine the effectiveness of control. However, the existing technical system is unable to accurately capture and correlate these core elements.

[0003] In terms of positioning technology, single UWB is prone to non-line-of-sight propagation in areas with dense rebar or obstructed by metal components. Signal attenuation and multipath effects cause positioning deviations that exceed the tolerance range of process acceptance. Although independent IMUs can collect motion data at high frequency, their accuracy drops sharply over long periods due to the cumulative effect of zero-bias drift, which cannot meet the needs of mobile supervision across work areas. In the field of behavior recognition, existing solutions are mostly limited to single posture sensing of the waist or head, which can only judge macroscopic states such as movement and stillness. They lack multidimensional perception of hand operation trajectory, muscle force patterns and tactile interaction intensity, making it difficult to distinguish easily confused actions such as bolt torque verification and formal touch inspection.

[0004] More importantly, existing technologies lack a correlation verification mechanism between location, action, and quality. Location data and behavioral data are isolated, making it impossible to form a closed-loop analysis of the spatial location and operational actions of supervisors with the quality parameters of key processes. This results in compliance judgments remaining at the rudimentary level of presence, making it difficult to deeply assess whether operations meet standards and quality requirements. Furthermore, dynamic interference in construction scenarios, such as electromagnetic radiation and vibration, coupled with the flexibility requirements of supervision operations, leads to problems such as data acquisition distortion and insufficient battery life for single-modal sensing devices. Multi-device collaboration, on the other hand, suffers from bottlenecks such as data synchronization delays and low fusion accuracy. These further restrict the transformation of supervision and control towards in-process intervention. These technological shortcomings result in a triple dilemma for digital supervision and control: inaccurate location leading to ambiguous responsibility definition, unclear behavioral semantics leading to biased compliance judgments, and lack of quality correlation leading to superficial control. Consequently, it fails to provide accurate process data support for engineering quality and safety.

[0005] In summary, existing technologies suffer from problems such as susceptibility to localization interference, coarse behavior recognition, broken quality correlation, and insufficient multi-modal fusion. Summary of the Invention

[0006] In order to overcome the problems of existing technologies such as easy interference in positioning, rough behavior recognition, broken quality correlation, and insufficient multi-mode fusion, this invention discloses a method and system for on-site positioning and behavior compliance analysis of supervisors based on the fusion of UWB and IMU, which can effectively solve the above-mentioned technical problems.

[0007] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0008] A method for analyzing the on-site location and behavioral compliance of supervisors based on the fusion of UWB and IMU, the method includes:

[0009] The supervisors are equipped with multimodal wearable sensing devices, which include a smart safety helmet node, a waist main control positioning node, dual wrist sensing nodes and finger cot tactile nodes, to collect the supervisors' UWB positioning signals, IMU motion data, surface electromyography signals and tactile pressure data.

[0010] A tightly coupled algorithm using error state Kalman filtering is employed to fuse UWB positioning signals and IMU motion data. A lightweight NLOS recognition network is introduced, and the reliability of observations is judged by the CIR characteristics of UWB signals and the consistency of IMU motion as inputs. The filter noise covariance matrix is ​​dynamically adjusted to achieve robust positioning and attitude estimation.

[0011] Spatial posture features, hand motor unit features, and head-gaze features are extracted from the collected data to construct a micro-motion feature vector. Spatial posture features include body orientation and movement state, hand motor unit features include IMU trajectory template matching results, electromyographic activation patterns, and tactile pressure curves, and head-gaze features are the global head orientation vector.

[0012] The semantic location labels mapped from micro-action feature vectors to UWB localization are input into the TP-LSTM-CRF model. The TP-LSTM captures short, medium, and long-term action dependencies, and the CRF layer optimizes the logicality of the behavior sequence, outputting a time-location-action semantic behavior flow with timestamps.

[0013] To build a digital twin model of the standard operating procedure for key supervision processes, define the sequence of necessary location points, the order of standard actions at each location, and the threshold values ​​of key action parameters;

[0014] Dynamically match semantic behavior streams with digital twin models to detect whether there are missing positions, missing actions, or incorrect sequences;

[0015] Based on data on movement amplitude, duration, and tactile pressure, the model parameter thresholds are compared to infer whether the operation quality meets the standard.

[0016] Integrate location results, behavior flow matching results, and quality inference conclusions to generate a preliminary compliance judgment and mark the time, location, and abnormal data of violations;

[0017] The above information is compiled into a visual analysis report, which includes a location trajectory map, compliance rate statistics, and rectification suggestions, thus completing the on-site location and behavioral compliance analysis of the supervisors.

[0018] Preferably, the configuration of the multimodal wearable sensing device includes:

[0019] The smart safety helmet node integrates a dual-antenna UWB module and a 9-axis IMU. The dual-antenna UWB module measures the head orientation by arriving phase difference, and the 9-axis IMU identifies the head posture.

[0020] The waist-mounted master positioning node includes a high-precision IMU, a UWB anchor communication module, and an edge computing unit. The high-precision IMU serves as the core of PDR to perceive body movement, while the edge computing unit runs a lightweight fusion algorithm and a basic behavior recognition program.

[0021] The dual wrist sensing nodes are equipped with a 9-axis IMU and a surface electromyography (EMG) sensor. The 9-axis IMU uses high-frequency sampling to capture hand movement trajectories, while the surface EMG sensor monitors forearm muscle electrical signals to distinguish force application patterns.

[0022] The finger-type tactile nodes are worn on the thumb and index finger and include miniature pressure sensors and bending sensors to measure gripping pressure and finger joint bending angle, respectively.

[0023] Preferably, the process of fusing data using the tightly coupled algorithm includes:

[0024] Position, velocity, attitude, and IMU zero bias are set as the state variables of the error state Kalman filter, and the raw arrival time of UWB and IMU acceleration and angular velocity are set as the observations.

[0025] The NLOS identification network analyzes the consistency between the CIR characteristic waveform of the UWB signal and the IMU motion online, and outputs the reliability level of the UWB observations.

[0026] The noise covariance matrix of UWB observations in the filter is dynamically adjusted according to the reliability level. High reliability observations correspond to low noise covariance, and low reliability observations correspond to high noise covariance, thereby improving positioning stability in complex environments.

[0027] Preferably, the construction of the micro-motion feature vector includes:

[0028] Spatial attitude features are calculated from waist IMU data and UWB positioning results. The body orientation angle is obtained through attitude calculation, and the static or moving state is determined by combining the motion speed.

[0029] In the hand motion unit features, IMU data is used to complete gesture trajectory template matching through the DTW algorithm to extract motion amplitude spectrum energy;

[0030] Surface electromyography (EMG) signals were used to extract muscle activation levels, activation duration, and multi-channel signal synergistic patterns.

[0031] Tactile data was used to extract the curve of peak pressure value versus pressure duration.

[0032] The head-gaze feature is obtained by fusing the UWB head orientation data of the smart helmet node with the IMU attitude data, and converting it into a head orientation vector in the global coordinate system to infer the direction of the gaze focus.

[0033] Preferably, the operation of the TP-LSTM-CRF model includes:

[0034] The model input layer synchronously receives micro-motion feature vectors and semantic location labels. The semantic location labels are generated by UWB positioning coordinate mapping and contain scene information.

[0035] The TP-LSTM layer captures the dependencies of short-term actions, medium-term action sequences, and long-term workflows through multi-scale time windows.

[0036] The CRF output layer imposes global constraints on the action sequences identified by the model, eliminates logically contradictory sequences, and ensures the time rationality of the behavior flow.

[0037] The model output format is a correspondence between time interval, semantic location, and executed action.

[0038] Preferably, the construction of the digital twin model of the standard operating procedure includes:

[0039] Based on construction drawings, safety regulations, and work instructions, a three-dimensional digital twin scene is built in the cloud, covering the entire process of key procedures;

[0040] Define the sequence of necessary locations for each process, and mark the spatial coordinate range and minimum dwell time requirement for each location;

[0041] Specify the standard sequence of actions to be performed at each location point and their order, and set parameter thresholds for key actions.

[0042] Preferably, the dynamic matching of the semantic behavior flow with the digital twin model includes:

[0043] Compare the location information in the semantic behavior flow with the sequence of necessary location points in the model. If any location point is missing and the dwell time requirement is not met, it is marked as a location omission.

[0044] Compare the action sequence of the semantic behavior flow with the model's standard action sequence. If there are missing actions or the order does not conform to the standard, it is marked as an action violation.

[0045] The deviation between the dwell time and action execution time at each position in the statistical semantic behavior flow and the model's predicted time is counted. When the deviation exceeds a preset threshold, it is marked as an anomaly in time consumption.

[0046] Preferably, the inference of operational quality includes:

[0047] Extract the amplitude, duration, and finger tactile pressure data of actions from the semantic behavior flow to form a quality analysis dataset;

[0048] The dataset is compared with the threshold values ​​of key motion parameters in the digital twin model. If the motion amplitude is within the threshold range, the pressure value meets the standard, and the duration meets the requirements, the operation quality is deemed qualified.

[0049] If the data exceeds the threshold range, it is marked as substandard and the specific abnormal data is recorded.

[0050] Preferably, the generation of the compliance analysis report includes:

[0051] The report includes a real-time location tracking map of the supervisors, with different colors used to mark compliant and missed locations, and the time spent at each location.

[0052] Present a complete list of semantic behavior flows, using different identifiers to mark violations and compliant actions, and attaching time nodes and screenshots of abnormal data for the violations;

[0053] Calculate the compliance rate, which is the ratio of compliant actions to the total number of actions. Also, count the number of times the position was missed, the number of times the sequence was incorrect, and the number of times the quality was substandard.

[0054] Suggestions for rectification were put forward for the violations.

[0055] Preferably, the system is a site location and behavior compliance analysis system for supervisors based on the fusion of UWB and IMU, the system comprising:

[0056] The multimodal sensing layer consists of a smart helmet node, a waist main control positioning node, dual wrist sensing nodes, and finger-cot tactile nodes, and is responsible for collecting UWB, IMU, surface electromyography, and tactile data.

[0057] The edge processing layer, deployed at the 5G / MEC gateway on the construction site, runs a tightly coupled fusion algorithm and an NLOS recognition network, and outputs robust localization results and micro-motion feature vectors.

[0058] The model computation layer includes TP-LSTM-CRF model units and digital twin modeling units. The former outputs semantic behavior flow, while the latter constructs a digital model of standard operating procedures.

[0059] The compliance analysis layer includes a dynamic rule matching module and a quality inference module. The dynamic rule matching module detects location / action violations, and the quality inference module judges the quality of operations and generates compliance results.

[0060] The results output layer aggregates location data, behavior flow, and compliance results to generate visual analysis reports, supporting data export, anomaly alert push notifications, and historical data queries.

[0061] Compared with existing technologies, the beneficial effects of this invention are as follows: This technical solution addresses the problem of positioning being susceptible to interference. It employs a tightly coupled algorithm of error-state Kalman filtering to fuse UWB positioning signals and IMU motion data. Simultaneously, a lightweight NLOS recognition network is introduced. The reliability of observations is judged by using the CIR characteristics of the UWB signal and the consistency of IMU motion as inputs, and the filter noise covariance matrix is ​​dynamically adjusted. This design enables the system to identify low-reliability UWB observations in real time and reduce their weight when encountering interference such as signal obstruction and multipath effects in complex construction environments. This avoids the influence of interference data on the positioning results, ultimately achieving robust positioning and attitude estimation, and solving the problems of traditional... To address the issue of drastically reduced accuracy in non-line-of-sight scenarios, and to mitigate the coarseness of behavior recognition, the solution first collects UWB positioning signals, IMU motion data, surface electromyography (EMG) signals, and tactile pressure data using a multimodal wearable sensing device. Then, it extracts spatial posture features (including body orientation and movement state), hand motion unit features (including IMU trajectory template matching results, EMG activation patterns, and tactile pressure curves), and head-gaze features (global head orientation vector) from the data to construct micro-motion feature vectors. Finally, the semantic location labels mapped to these vectors by UWB positioning are input into the TP-LSTM-CRF model, which captures multi-dimensional features and utilizes multi-scale time windows for motion analysis. Dependency analysis, combined with global constraints on behavior sequences at the CRF layer, enables the system to output refined time-location-action semantic behavior flows with timestamps. This overcomes the limitations of traditional technologies, which can only identify simple actions and cannot correlate location and action logic. Addressing the issue of quality correlation breakdown, the solution first constructs a digital twin model of the standard operating procedure for key supervisory processes, defining the sequence of necessary location points, the order of standard actions at each location, and threshold values ​​for key action parameters. Then, the semantic behavior flow is dynamically matched with the digital twin model. Simultaneously, based on the comparison of action amplitude, duration, and tactile pressure data with the model parameter thresholds, by directly associating behavioral data with preset quality standards, the system can accurately infer the operational... This solution addresses the challenge of ensuring quality standards are met, resolving the difficulties in traditional technologies where behavior recording and quality assessment are independent and the source of quality issues cannot be traced. To address the insufficient multimodal fusion, the solution employs a multimodal sensing layer (including nodes such as smart helmets and waist control units) to collect various types of data. An edge processing layer runs a tightly coupled fusion algorithm and an NLOS recognition network to achieve initial data fusion. The model computation layer and compliance analysis layer further complete feature fusion and result fusion. Through collaboration across all layers, deep multimodal data fusion is achieved throughout the entire process from data acquisition to result output. This avoids the information waste caused by isolated multi-sensor data and low fusion levels in traditional technologies, improving the overall analytical accuracy and reliability of the system. Attached Figure Description

[0062] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other embodiments can be derived from the provided drawings without creative effort.

[0063] Figure 1 This is a diagram illustrating the steps of the method of the present invention;

[0064] Figure 2 This is a hierarchical framework diagram of the system of the present invention. Detailed Implementation

[0065] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0066] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions;

[0067] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0068] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0069] Example 1

[0070] In a large-scale construction project, to ensure the efficiency and standardization of the supervision work, a method for analyzing the on-site positioning and behavioral compliance of supervisors based on the fusion of UWB and IMU was introduced. The project site covers a vast area with a complex construction environment, including multiple construction areas and different types of building structures, which places high demands on the positioning accuracy and behavioral monitoring of supervisors.

[0071] Please see Figure 1 A method for analyzing on-site location and behavioral compliance of supervisors based on the fusion of UWB and IMU, the method comprising:

[0072] The supervisors are equipped with multimodal wearable sensing devices, which include a smart safety helmet node, a waist main control positioning node, dual wrist sensing nodes and finger cot tactile nodes, to collect the supervisors' UWB positioning signals, IMU motion data, surface electromyography signals and tactile pressure data.

[0073] A tightly coupled algorithm using error state Kalman filtering is employed to fuse UWB positioning signals and IMU motion data. A lightweight NLOS recognition network is introduced, and the reliability of observations is judged by the CIR characteristics of UWB signals and the consistency of IMU motion as inputs. The filter noise covariance matrix is ​​dynamically adjusted to achieve robust positioning and attitude estimation.

[0074] Spatial posture features, hand motor unit features, and head-gaze features are extracted from the collected data to construct a micro-motion feature vector. Spatial posture features include body orientation and movement state, hand motor unit features include IMU trajectory template matching results, electromyographic activation patterns, and tactile pressure curves, and head-gaze features are the global head orientation vector.

[0075] The semantic location labels mapped from micro-action feature vectors to UWB localization are input into the TP-LSTM-CRF model. The TP-LSTM captures short, medium, and long-term action dependencies, and the CRF layer optimizes the logicality of the behavior sequence, outputting a time-location-action semantic behavior flow with timestamps.

[0076] To build a digital twin model of the standard operating procedure for key supervision processes, define the sequence of necessary location points, the order of standard actions at each location, and the threshold values ​​of key action parameters;

[0077] Dynamically match semantic behavior streams with digital twin models to detect whether there are missing positions, missing actions, or incorrect sequences;

[0078] Based on data on movement amplitude, duration, and tactile pressure, the model parameter thresholds are compared to infer whether the operation quality meets the standard.

[0079] Integrate location results, behavior flow matching results, and quality inference conclusions to generate a preliminary compliance judgment and mark the time, location, and abnormal data of violations;

[0080] The above information is compiled into a visual analysis report, which includes a location trajectory map, compliance rate statistics, and rectification suggestions, thus completing the on-site location and behavioral compliance analysis of the supervisors.

[0081] The configuration of the multimodal wearable sensing device includes:

[0082] The smart safety helmet node integrates a dual-antenna UWB module and a 9-axis IMU. The dual-antenna UWB module measures the head orientation by arriving phase difference, and the 9-axis IMU identifies the head posture.

[0083] The waist-mounted master positioning node includes a high-precision IMU, a UWB anchor communication module, and an edge computing unit. The high-precision IMU serves as the core of PDR to perceive body movement, while the edge computing unit runs a lightweight fusion algorithm and a basic behavior recognition program.

[0084] The dual wrist sensing nodes are equipped with a 9-axis IMU and a surface electromyography (EMG) sensor. The 9-axis IMU uses high-frequency sampling to capture hand movement trajectories, while the surface EMG sensor monitors forearm muscle electrical signals to distinguish force application patterns.

[0085] The finger-type tactile nodes are worn on the thumb and index finger and include miniature pressure sensors and bending sensors to measure gripping pressure and finger joint bending angle, respectively.

[0086] The process of data fusion using the tightly coupled algorithm includes:

[0087] Position, velocity, attitude, and IMU zero bias are set as the state variables of the error state Kalman filter, and the raw arrival time of UWB and IMU acceleration and angular velocity are set as the observations.

[0088] The NLOS identification network analyzes the consistency between the CIR characteristic waveform of the UWB signal and the IMU motion online, and outputs the reliability level of the UWB observations.

[0089] The noise covariance matrix of UWB observations in the filter is dynamically adjusted according to the reliability level. High reliability observations correspond to low noise covariance, and low reliability observations correspond to high noise covariance, thereby improving positioning stability in complex environments.

[0090] The construction of the micro-action feature vector includes:

[0091] Spatial attitude features are calculated from waist IMU data and UWB positioning results. The body orientation angle is obtained through attitude calculation, and the static or moving state is determined by combining the motion speed.

[0092] In the hand motion unit features, IMU data is used to complete gesture trajectory template matching through the DTW algorithm to extract motion amplitude spectrum energy;

[0093] Surface electromyography (EMG) signals were used to extract muscle activation levels, activation duration, and multi-channel signal synergistic patterns.

[0094] Tactile data was used to extract the curve of peak pressure value versus pressure duration.

[0095] The head-gaze feature is obtained by fusing the UWB head orientation data of the smart helmet node with the IMU attitude data, and converting it into a head orientation vector in the global coordinate system to infer the direction of the gaze focus.

[0096] The operation of the TP-LSTM-CRF model includes:

[0097] The model input layer synchronously receives micro-motion feature vectors and semantic location labels. The semantic location labels are generated by UWB positioning coordinate mapping and contain scene information.

[0098] The TP-LSTM layer captures the dependencies of short-term actions, medium-term action sequences, and long-term workflows through multi-scale time windows.

[0099] The CRF output layer imposes global constraints on the action sequences identified by the model, eliminates logically contradictory sequences, and ensures the time rationality of the behavior flow.

[0100] The model output format is a correspondence between time interval, semantic location, and executed action.

[0101] The construction of the digital twin model of the standard operating procedure includes:

[0102] Based on construction drawings, safety regulations, and work instructions, a three-dimensional digital twin scene is built in the cloud, covering the entire process of key procedures;

[0103] Define the sequence of necessary locations for each process, and mark the spatial coordinate range and minimum dwell time requirement for each location;

[0104] Specify the standard sequence of actions to be performed at each location point and their order, and set parameter thresholds for key actions.

[0105] The dynamic matching of the semantic behavior flow with the digital twin model includes:

[0106] Compare the location information in the semantic behavior flow with the sequence of necessary location points in the model. If any location point is missing and the dwell time requirement is not met, it is marked as a location omission.

[0107] Compare the action sequence of the semantic behavior flow with the model's standard action sequence. If there are missing actions or the order does not conform to the standard, it is marked as an action violation.

[0108] The deviation between the dwell time and action execution time at each position in the statistical semantic behavior flow and the model's predicted time is counted. When the deviation exceeds a preset threshold, it is marked as an anomaly in time consumption.

[0109] The inference of operational quality includes:

[0110] Extract the amplitude, duration, and finger tactile pressure data of actions from the semantic behavior flow to form a quality analysis dataset;

[0111] The dataset is compared with the threshold values ​​of key motion parameters in the digital twin model. If the motion amplitude is within the threshold range, the pressure value meets the standard, and the duration meets the requirements, the operation quality is deemed qualified.

[0112] If the data exceeds the threshold range, it is marked as substandard and the specific abnormal data is recorded.

[0113] The generation of the compliance analysis report includes:

[0114] The report includes a real-time location tracking map of the supervisors, with different colors used to mark compliant and missed locations, and the time spent at each location.

[0115] Present a complete list of semantic behavior flows, using different identifiers to mark violations and compliant actions, and attaching time nodes and screenshots of abnormal data for the violations;

[0116] Calculate the compliance rate, which is the ratio of compliant actions to the total number of actions. Also, count the number of times the position was missed, the number of times the sequence was incorrect, and the number of times the quality was substandard.

[0117] Suggestions for rectification were put forward for the violations.

[0118] Equip supervisors with multimodal wearable sensing devices, including smart safety helmet nodes, waist main control positioning nodes, dual wrist sensing nodes, and finger cot tactile nodes. These devices work together to comprehensively collect various types of data from supervisors.

[0119] The smart safety helmet integrates a dual-antenna UWB module and a 9-axis IMU. On the construction site, the dual-antenna UWB module measures head orientation by arriving at the phase difference. When supervisors patrol different construction areas, this module can sense the direction of head rotation in real time, providing directional information for behavior analysis. The 9-axis IMU can accurately identify head posture, whether the supervisor is looking up to check the construction situation at height or looking down to record data, it can be accurately captured. For example, when inspecting the construction quality of the building facade, supervisors need to frequently look up to observe the flatness of the wall and construction details. The 9-axis IMU can convert changes in head posture into data in a timely manner, providing a basis for analyzing the supervisors' work focus.

[0120] The waist-mounted master positioning node includes a high-precision IMU, a UWB anchor communication module, and an edge computing unit. The high-precision IMU, as the core of pedestrian trajectory estimation (PDR), can sense body movement. When supervisors move around the construction site, it analyzes data such as acceleration and angular velocity to calculate the number of steps, step length, and direction of travel, thus achieving continuous position tracking. The UWB anchor communication module communicates with UWB anchors distributed throughout the construction site to obtain accurate positioning information, ensuring positioning accuracy. The edge computing unit runs lightweight fusion algorithms and basic behavior recognition programs to perform preliminary processing and analysis of the collected data, reducing the pressure on subsequent data processing and improving the system's response speed. For example, when supervisors quickly move from one construction area to another, the waist-mounted master positioning node can rapidly process IMU and UWB data, update their position information in a timely manner, and make a preliminary judgment on their movement status, providing data for behavior analysis.

[0121] The dual wrist sensing nodes are equipped with a 9-axis IMU and a surface electromyography (EMG) sensor. The 9-axis IMU uses high-frequency sampling to accurately capture hand movement trajectories. When supervisors are recording or inspecting tools, it can record detailed information such as the hand's movement path, speed, and acceleration. The EMG sensor monitors the electrical signals of the forearm muscles. By analyzing these signals, force application patterns can be distinguished. For example, when supervisors use different grip strengths on tools, the EMG sensor can detect changes in the electrical signals of the muscles, thereby determining the magnitude and manner of force application and providing data for analyzing the standardization and rationality of their operations.

[0122] The finger-type tactile node is worn on the thumb and index finger and includes a miniature pressure sensor and a bending sensor. When the supervisor grasps an object or performs a fine operation, the miniature pressure sensor can measure the gripping pressure, while the bending sensor can detect the bending angle of the finger joint. For example, when inspecting small building materials, the supervisor needs to grasp and touch the materials with their fingers. The finger-type tactile node can record the gripping force and the degree of finger bending in real time, providing data to assess the stability and accuracy of their operation, and thus determine whether their operation meets the specifications.

[0123] At the construction site, multimodal wearable sensing devices comprehensively collect UWB positioning signals, IMU motion data, surface electromyography (EMG) signals, and tactile pressure data from the supervisors. The UWB positioning signals are acquired through communication between the UWB modules of the smart safety helmet node and the waist master control positioning node and UWB anchor points distributed throughout the construction site, providing the supervisors with location coordinates and accurately determining their specific location. IMU motion data is collected by the IMUs of the smart safety helmet node, waist master control positioning node, and dual wrist sensing nodes, reflecting the supervisors' body movement status and posture changes in real time. Surface EMG signals are collected by the surface EMG sensors of the dual wrist sensing nodes, used to analyze the supervisors' hand muscle activity and understand their force exertion during operation. Tactile pressure data is collected by finger-type tactile nodes, recording the pressure and bending information of the fingers during operation, providing data for behavioral analysis.

[0124] A tightly coupled algorithm using error-state Kalman filtering is employed to fuse UWB positioning signals and IMU motion data. Position, velocity, attitude, and IMU zero bias are set as state variables of the error-state Kalman filter, while the raw UWB arrival time and IMU acceleration and angular velocity are set as observations. During actual operation, the filter predicts the current state based on the state of the previous moment and updates it with the latest observation data. For example, when a supervisor walks on the construction site, the filter predicts the changes in their position and velocity based on the acceleration and angular velocity measured by the IMU, and corrects them by combining the UWB positioning signal, thereby obtaining a more accurate positioning result. This fusion method fully utilizes the high precision of UWB positioning and the autonomy of the IMU, improving the stability and reliability of positioning.

[0125] A lightweight NLOS recognition network is introduced, using the CIR characteristics of UWB signals and IMU motion consistency as inputs to determine the reliability of observations. In complex construction sites, UWB signals are easily affected by obstruction and multipath interference, leading to signal quality degradation. The NLOS recognition network analyzes the CIR characteristic waveform of UWB signals to determine whether the signal is affected by non-line-of-sight (NLOS) interference, and combines this with the consistency of IMU motion data to further determine the reliability of observations. For example, when supervisors walk inside buildings, UWB signals may be obstructed by walls or other obstacles. The NLOS recognition network can accurately identify this situation and dynamically adjust the filter noise covariance matrix. For high-reliability observations, a low noise covariance is used, making the filter more reliant on these reliable data for positioning; for low-reliability observations, a high noise covariance is used, reducing their impact on positioning results, thereby improving the stability of positioning in complex environments.

[0126] Spatial posture features, hand motion unit features, and head-line features are extracted from the collected data to construct micro-motion feature vectors. Spatial posture features are calculated from waist IMU data and UWB positioning results. The body orientation angle is obtained through posture calculation, and the stationary or moving state is determined by combining the movement speed. When the supervisor stands and observes at the construction site, the body orientation angle and movement speed data can indicate the direction of their attention and whether they are stationary, providing a basis for analyzing their working status.

[0127] The process of extracting hand motion unit features involves using IMU data to perform gesture trajectory template matching via the DTW algorithm to extract motion amplitude spectrum energy. When supervisors make different gestures, such as indicating or recording, the DTW algorithm can match the real-time acquired IMU data with predefined gesture templates to determine the gesture type and extract motion amplitude spectrum energy, reflecting the intensity and changes of the gesture. Surface electromyography (SEMG) signals are used to extract muscle activation levels, activation duration, and multi-channel signal coordination patterns. Different hand operations lead to different activation patterns of forearm muscles. By analyzing these characteristics of SEMG signals, we can understand the force application method and duration of the supervisor's hand operations. Tactile data is used to extract peak pressure values ​​and pressure duration curves. When supervisors grasp tools or objects, the pressure data collected by the finger-type tactile nodes can reflect the gripping force and duration, providing data for analyzing the stability of their operations.

[0128] The head-to-eye feature is obtained by fusing the UWB head orientation data of the smart safety helmet node with the IMU attitude data and converting it into a head orientation vector in the global coordinate system. This allows the inference of the direction of the gaze focus. On the construction site, the gaze focus direction of the supervisor can reflect the area of ​​concern and the focus of the work. For example, when the supervisor looks up at the construction site at a high position, the head-to-eye feature can accurately determine the direction of his gaze, providing important information for analyzing his work behavior.

[0129] The micro-motion feature vectors and semantic location labels mapped to UWB positioning are input into the TP-LSTM-CRF model. The model input layer simultaneously receives the micro-motion feature vectors and semantic location labels. The semantic location labels are generated by UWB positioning coordinate mapping and contain scene information, such as construction area and floor. The TP-LSTM layer captures the dependencies of short-term actions, medium-term action sequences, and long-term work processes through multi-scale time windows. For example, for short-term actions, such as a supervisor's quick gesture, the TP-LSTM layer can capture its instantaneous action features through a small time window; for medium-term action sequences, such as a supervisor's continuous recording operations over a period of time, the TP-LSTM layer can analyze the order and correlation between actions; for long-term work processes, such as a supervisor's full supervision of a construction stage, the TP-LSTM layer can grasp the key steps and time sequence of the entire process.

[0130] The CRF output layer globally constrains the action sequences identified by the model, eliminating logically contradictory sequences and ensuring the temporal rationality of the behavior flow. In practical applications, the CRF output layer optimizes the action sequences output by the TP-LSTM layer based on the logical relationships and temporal order between actions. For example, when the TP-LSTM layer identifies that the action sequence of the supervisor is not illogical, such as recording before inspection, the CRF output layer will adjust the action sequence according to predefined behavioral logic rules to ensure that the output behavior flow conforms to the actual workflow. The final output format of the model is the correspondence between time interval, semantic location, and executed actions, providing a clear data structure for behavior analysis and compliance judgment.

[0131] Based on construction drawings, safety regulations, and work instructions, a 3D digital twin scene was built in the cloud, covering the entire process of key procedures. In this construction project, the digital twin scene detailed the layout of each construction area, the building structure, and the location of construction equipment. It defined the sequence of necessary locations for each procedure, marked the spatial coordinate range and minimum dwell time requirements for each location. For example, in the concrete pouring process, supervisors need to inspect and supervise at multiple locations such as the mixing plant and the pouring site. The digital twin model clearly specifies the specific coordinate range of these locations and the minimum dwell time for supervisors at each location to ensure the comprehensiveness and effectiveness of the supervision work. It also specifies the standard action sequence and order to be performed at each location and sets parameter thresholds for key actions. When inspecting the quality of building materials, supervisors need to perform standard actions such as checking material labels, inspecting appearance, and sampling tests at specific locations. The digital twin model clearly defines the order of these actions and sets parameter thresholds for key actions, such as sampling quantity and testing time, to provide standards for judging the compliance of supervisors' behavior.

[0132] The system dynamically matches semantic behavior flows with digital twin models to detect omissions in location, missing actions, or incorrect sequences. It compares the location information in the semantic behavior flow with the required sequence of location points in the model. If any location point is missing and the dwell time requirement is not met, it is marked as a location omission. For example, in the steel structure installation process, if the supervisor fails to reach the designated location point to inspect the steel beam connection and the dwell time is insufficient, the system will automatically mark it as a location omission and record the relevant information. The system also compares the action sequence of the semantic behavior flow with the standard action sequence of the model. If there are missing actions or the sequence does not conform to the standard, it is marked as an action violation. For example, in the electrical equipment installation inspection, if the supervisor does not perform insulation testing, grounding inspection, or other operations according to the standard action sequence, or the action sequence is incorrect, the system will promptly mark it as an action violation. The system also calculates the deviation between the dwell time and action execution time at each location in the semantic behavior flow and the model's estimated time consumption. When the deviation exceeds a preset threshold, it is marked as an abnormal time consumption. In this way, the system can comprehensively and accurately detect the differences between the supervisor's behavior and the standard operating procedures, promptly identify problems, and correct them.

[0133] The amplitude, duration, and tactile pressure data of actions in the semantic behavior flow are extracted to form a quality analysis dataset. This dataset is then compared with the key action parameter thresholds in the digital twin model to determine whether the operation quality meets the standards. When inspecting the verticality of building walls, the amplitude, duration, and finger pressure data of the supervisor using the measuring tool are all closely related to the operation quality. If the amplitude is within the threshold range, it indicates that the supervisor's operation is standardized and accurate. If the pressure value meets the standard and the duration meets the requirements, it indicates that the supervisor can hold the measuring tool stably and ensure the accuracy of the measurement results. If the data exceeds the threshold range, it is marked as substandard, and the specific abnormal data is recorded for subsequent analysis and rectification.

[0134] Integrating location results, behavior flow matching results, and quality inference conclusions, a visual analysis report is generated. The report includes a real-time location trajectory map of the supervisors, using different colors to mark compliant and missed locations, and indicating the dwell time at each location. On the location trajectory map, compliant locations are displayed in green, and missed locations in red, with clear indication of the dwell time at each location. This allows managers to intuitively understand the supervisors' movement paths and work status. A complete semantic behavior flow list is presented, using different icons to mark compliant and non-compliant actions, and including time nodes and screenshots of abnormal data for non-compliant actions. In the semantic behavior flow list, non-compliant actions are marked with color or symbols, along with the time nodes of the non-compliant actions and related abnormal data screenshots, facilitating managers' detailed understanding of violations. The compliance rate is calculated as the ratio of compliant actions to total actions. The report also statistically analyzes the number of missed locations, the number of incorrect sequences, and the number of quality deficiencies. Corrective action suggestions are proposed for violations, such as strengthening training and optimizing workflows, providing specific improvement directions for enhancing the quality of supervision work.

[0135] Example 2

[0136] Please see Figure 2 A system for analyzing the on-site location and behavioral compliance of supervisors based on the fusion of UWB and IMU, the system comprising:

[0137] The multimodal sensing layer consists of a smart helmet node, a waist main control positioning node, dual wrist sensing nodes, and finger-cot tactile nodes, and is responsible for collecting UWB, IMU, surface electromyography, and tactile data.

[0138] The edge processing layer, deployed at the 5G / MEC gateway on the construction site, runs a tightly coupled fusion algorithm and an NLOS recognition network, and outputs robust localization results and micro-motion feature vectors.

[0139] The model computation layer includes TP-LSTM-CRF model units and digital twin modeling units. The former outputs semantic behavior flow, while the latter constructs a digital model of standard operating procedures.

[0140] The compliance analysis layer includes a dynamic rule matching module and a quality inference module. The dynamic rule matching module detects location / action violations, and the quality inference module judges the quality of operations and generates compliance results.

[0141] The results output layer aggregates location data, behavior flow, and compliance results to generate visual analysis reports, supporting data export, anomaly alert push notifications, and historical data queries.

[0142] The multimodal sensing layer, serving as the data acquisition source, is composed of smart safety helmet nodes, waist main control positioning nodes, wrist sensing nodes, and finger cot tactile nodes. These nodes work closely together to collect various key data from supervisors at the work site, providing data for analysis and decision-making.

[0143] The smart safety helmet integrates a dual-antenna UWB module and a 9-axis IMU. In actual construction scenarios, when supervisors move between different construction areas, the dual-antenna UWB module, with its arrival phase difference measurement technology, can accurately sense the direction of their head rotation. For example, in high-altitude work areas on construction sites, supervisors need to frequently turn their heads to observe whether workers' operations are standardized and whether safety protection measures are in place. The dual-antenna UWB module can capture this head rotation information in real time, providing a basis for analyzing the supervisors' work focus. Meanwhile, the 9-axis IMU can accurately identify head posture in real time. Whether looking up to check high-altitude construction details or looking down to record data, it can be accurately captured, thus comprehensively reflecting the supervisors' head movement status.

[0144] The waist-mounted master positioning node is equipped with a high-precision IMU, a UWB anchor communication module, and an edge computing unit. The high-precision IMU, as the core component of pedestrian trajectory estimation (PDR), can sensitively sense the body's movement state. When the supervisor walks on the construction site, it accurately calculates the number of steps, step length, and direction of travel through real-time analysis of data such as acceleration and angular velocity, thereby achieving continuous tracking of the supervisor's position. The UWB anchor communication module is responsible for efficient communication with UWB anchors pre-deployed on the construction site to obtain accurate positioning information, ensuring that the accuracy of positioning is not affected by the environment. The edge computing unit runs a lightweight fusion algorithm and basic behavior recognition program to perform preliminary processing and analysis on the collected data, reducing the pressure of subsequent data processing, improving the system's response speed, and enabling the system to promptly provide feedback on the supervisor's position and movement information.

[0145] The dual wrist sensing nodes are equipped with a 9-axis IMU and a surface electromyography (EMG) sensor. The 9-axis IMU, through high-frequency sampling, can accurately capture the movement trajectory of the hand. When supervisors perform routine operations such as recording and inspecting tools, it can record detailed information such as the hand's movement path, speed, and acceleration, providing detailed data for analyzing their operational behavior. The EMG sensor focuses on monitoring the electrical signals of the forearm muscles. Through in-depth analysis of these signals, it can accurately distinguish the force application mode. For example, when supervisors use different grip strengths on tools, the EMG sensor can sensitively detect changes in muscle electrical signals, thereby determining the magnitude and method of force application, providing key data for assessing the standardization and rationality of their operations.

[0146] The finger-type tactile node, worn on the thumb and index finger, includes a miniature pressure sensor and a bending sensor. When supervisors grasp objects or perform fine operations, the miniature pressure sensor can measure the gripping pressure in real time, while the bending sensor can accurately detect the bending angle of the finger joints. For example, when inspecting small building materials, supervisors need to grasp and touch the materials with their fingers. The finger-type tactile node can record the gripping force and the degree of finger bending in a timely and accurate manner, providing data for assessing the stability and accuracy of their operations, and thus helping to determine whether their operations meet the specifications. Through the collaborative work of various nodes in the multimodal sensing layer, comprehensive and accurate data on the supervisors' work status can be collected.

[0147] The edge processing layer, deployed on the 5G / MEC gateway at the construction site, is a key link in the data processing and analysis of the entire system. It undertakes the tasks of data fusion and feature extraction. Its core function is to run tightly coupled fusion algorithms and NLOS recognition networks. Through these algorithms and technologies, it efficiently processes the data collected by the multimodal sensing layer and outputs robust positioning results and micro-motion feature vectors, providing high-quality data for model calculation and compliance analysis.

[0148] The tightly coupled fusion algorithm employs an error-state Kalman filter, setting position, velocity, attitude, and IMU zero bias as state variables for the error-state Kalman filter, and the raw UWB arrival time, IMU acceleration, and angular velocity as observations. During actual operation, the algorithm predicts the current state based on the previous state and updates it in real-time using the latest observation data. For example, when supervisors move rapidly on the construction site, the filter quickly predicts their position and velocity changes based on the acceleration and angular velocity measured by the IMU, while simultaneously using the UWB positioning signal for precise correction, resulting in more accurate and reliable positioning results. This fusion method fully leverages the high precision of UWB positioning and the autonomy of the IMU, improving the stability and reliability of positioning and ensuring accurate tracking of supervisors' positions even in complex and ever-changing construction site environments.

[0149] NLOS identification network is another key technology in the edge processing layer. In complex construction sites, UWB signals are easily affected by obstruction and multipath interference, leading to signal quality degradation and affecting positioning accuracy. The NLOS identification network accurately determines whether the signal is affected by non-line-of-sight (NLOS) interference by analyzing the CIR characteristic waveform of the UWB signal in real time online. Combined with the consistency of IMU motion data, it further determines the reliability of the observations. For example, when supervisors walk inside a building, the UWB signal may be obstructed by obstacles such as walls and equipment. The NLOS identification network can quickly identify this situation and dynamically adjust the filter noise covariance matrix according to the signal reliability. For high-reliability observations, it corresponds to low noise covariance, making the filter more reliant on these reliable data for positioning; for low-reliability observations, it corresponds to high noise covariance, reducing their impact on the positioning results, thereby improving the stability of positioning in complex environments and ensuring the accuracy and reliability of the positioning results.

[0150] Through the collaborative processing of the tightly coupled fusion algorithm and the NLOS recognition network, the edge processing layer successfully outputs robust positioning results and micro-motion feature vectors. These data not only accurately reflect the location information of the supervisors, but also contain rich motion features, providing high-quality data input for the model calculation layer. This is an important guarantee for achieving accurate behavior analysis and compliance judgment. The efficient operation of the edge processing layer reduces the pressure of subsequent data processing and improves the overall operating efficiency and response speed of the system.

[0151] The model computation layer mainly consists of TP-LSTM-CRF model units and digital twin modeling units. These two units work together to complete in-depth analysis of the behavior of supervisors and modeling and comparing standard operating procedures, providing support for compliance analysis.

[0152] The TP-LSTM-CRF model unit undertakes the key tasks of behavior recognition and semantic behavior flow generation in the system. Its workflow starts with receiving the micro-action feature vector and semantic location label output by the edge processing layer. The semantic location label is generated by UWB positioning coordinate mapping and contains rich scene information, such as construction area, floor, specific work location, etc. This information provides background information for behavior analysis.

[0153] The TP-LSTM layer is one of the core components of this model unit. Through the design of multi-scale time windows, it captures the dependencies of short-term actions, medium-term action sequences, and long-term work processes. For short-term actions, such as a supervisor's quick gesture, the TP-LSTM layer can quickly capture the instantaneous action features using a small time window, accurately identifying the type and meaning of the gesture. For medium-term action sequences, such as a supervisor's continuous recording operations over a period of time, the TP-LSTM layer can deeply analyze the order and relationship between actions, understanding its workflow and intent. For long-term work processes, such as a supervisor's full-process supervision of a construction stage, the TP-LSTM layer can comprehensively grasp the key steps and time sequence of the entire process, judging the completeness and standardization of the work.

[0154] The CRF output layer performs global constraints and optimizations on the action sequences identified by the TP-LSTM layer. In actual work scenarios, the behavior sequences of supervisors need to conform to certain logic and time order. Based on this requirement, the CRF output layer conducts a detailed review and adjustment of the action sequences output by the TP-LSTM layer according to the logical relationships and time order between actions. For example, when the TP-LSTM layer identifies that the action sequences of supervisors are not in line with common sense, such as recording before checking, which is inconsistent with the normal supervisory workflow, the CRF output layer will reorder and correct the action sequences according to predefined behavioral logic rules to ensure that the output behavior flow conforms to the actual workflow and logical requirements.

[0155] After processing by the TP-LSTM-CRF model unit, the final output format is a semantic behavior flow with the correspondence between time interval, semantic location, and executed actions. This clear and accurate output format provides intuitive and effective data for compliance analysis, enabling analysts to understand at a glance the specific actions performed by supervisors at different times and locations, and providing a basis for judging whether their behavior is compliant.

[0156] The digital twin modeling unit builds a realistic 3D digital twin scene in the cloud based on important data such as construction drawings, safety regulations, and work instructions. It comprehensively covers the entire process of key procedures. This digital twin scene is like a virtual replica of the construction site, which presents in detail the layout of each construction area, the building structure, and the location of construction equipment, providing accurate standards and reference frameworks for the compliance analysis of the behavior of supervisors.

[0157] When constructing a digital twin model, it is first necessary to define the sequence of essential locations for each process and accurately mark the spatial coordinate range and minimum dwell time requirement for each location. Taking the concrete pouring process as an example, supervisors need to conduct rigorous inspections and supervision at multiple key locations such as the mixing plant and the pouring site. The digital twin model will clearly define the specific coordinate range of these locations to ensure that supervisors can accurately reach the designated locations to perform their work. At the same time, it will also set the minimum dwell time for supervisors at each location to ensure that they have enough time to conduct a comprehensive and detailed inspection, preventing important issues from being overlooked due to insufficient dwell time, thereby ensuring the comprehensiveness and effectiveness of the supervision work.

[0158] Furthermore, the digital twin modeling unit specifies the standard sequence and order of actions to be performed at each location point, and sets strict parameter thresholds for key actions. When inspecting the quality of building materials, supervisors need to perform actions such as checking material labels, inspecting appearance, and sampling tests at specific locations according to standard procedures. The digital twin model clarifies the order of these actions to ensure that supervisors operate in accordance with specifications. At the same time, it sets parameter thresholds for key actions, such as sampling quantity and testing time. These thresholds are determined based on engineering standards and experience and are used to judge whether the supervisors' operations meet the requirements. The standard operating procedure digital model constructed by the digital twin modeling unit provides clear and specific standards for judging the compliance of supervisors' behavior, making compliance analysis more scientific, accurate, and evidence-based.

[0159] The compliance analysis layer mainly consists of a dynamic rule matching module and a quality inference module. These two modules each perform their own functions, conducting a comprehensive and detailed analysis of the supervisors' behavior from different perspectives, thereby accurately determining whether their behavior is compliant and generating corresponding compliance results.

[0160] The core task of the dynamic rule matching module is to detect position and action violations in the semantic behavior flow. It discovers potential violations by dynamically and in real time matching the semantic behavior flow with the standard operating procedure digital model constructed by the digital twin modeling unit.

[0161] In terms of location violation detection, the dynamic rule matching module carefully compares the location information in the semantic behavior flow with the sequence of required location points specified in the model. If any location point is found to be missing in the semantic behavior flow and the minimum dwell time requirement set in the digital twin model is not met, the system will immediately mark it as a location omission. For example, in the steel structure installation process, the digital twin model clearly stipulates that the supervisor needs to check multiple locations such as the steel beam connection point. Each location point has a corresponding coordinate range and minimum dwell time. If the supervisor does not reach a certain specified steel beam connection point for inspection in actual work, or the dwell time at that location is insufficient, the dynamic rule matching module will quickly capture this anomaly and mark it as a location omission. At the same time, it will record relevant information, such as the missing location point and the actual dwell time, for further analysis and processing.

[0162] In terms of action violation detection, this module will conduct in-depth comparisons between the action sequence of the semantic behavior flow and the standard action sequence in the digital twin model. Once it finds that there are missing actions or that the action sequence does not conform to the standard, it will mark them as action violations. For example, in the inspection of electrical equipment installation, the standard action sequence requires the supervisor to perform insulation testing first and then grounding inspection. If it is found in the semantic behavior flow that the supervisor has not performed the insulation testing action, or has performed the grounding inspection first and then the insulation testing, the dynamic rule matching module will promptly identify these action violations and record the content of the violation action, the time of occurrence, and other information in detail, providing a basis for rectification and training.

[0163] In addition, the dynamic rule matching module will also count the deviation between the dwell time and action execution time at each position in the semantic behavior flow and the time expected by the digital twin model. When the deviation exceeds the preset threshold, it will be marked as an abnormal time consumption. This helps to discover potential work efficiency problems or unfamiliarity with operation by supervisors during the work process, and take timely measures to improve and optimize.

[0164] The quality inference module is mainly responsible for judging the quality of operation based on semantic behavior flow data and generating compliance results. Its workflow first extracts the amplitude value, duration and tactile pressure data collected by the finger tactile nodes of the semantic behavior flow. These data are then integrated to form a quality analysis dataset. This data can intuitively reflect key information such as the strength, time control and hand operation stability of the supervisor during the operation process, and is an important basis for judging the quality of operation.

[0165] The quality inference module performs a detailed comparison between the quality analysis dataset and the key action parameter thresholds set in the digital twin modeling unit. Taking the inspection of building wall verticality as an example, the range of motion, duration, and pressure data of the supervisor's fingers gripping the measuring tool are all closely related to the operation quality. The digital twin model sets reasonable threshold ranges for these parameters. For example, the range of motion should be within a certain range to ensure measurement accuracy; the pressure value should reach a certain standard to ensure stable grip of the measuring tool; and the duration should also meet the requirements to complete a comprehensive and accurate measurement. If the range of motion in the quality analysis dataset is within the threshold range, it indicates that the supervisor's operation is standardized and can perform accurate measurements; if the pressure value meets the standard and the duration meets the requirements, it indicates that the supervisor can hold the measuring tool stably, ensuring the reliability of the measurement results. At this point, the quality inference module will determine that the operation quality is qualified.

[0166] However, if the data exceeds the threshold range, the quality inference module will mark it as substandard and record the specific abnormal data in detail, such as the value of the actual movement range exceeding the range, the degree of insufficient pressure value, and the duration of too short or too long. These recorded abnormal data provide detailed information for quality problem analysis and rectification, which helps to take targeted measures to improve the operational quality and work level of the supervisors. Through the work of the quality inference module, the quantitative assessment of the operational quality of the supervisors and the judgment of compliance can be realized.

[0167] The output layer comprehensively summarizes location data, behavior flow, and compliance results, presenting them to relevant managers in the form of a visual analysis report. It also supports functions such as data export, anomaly alert push, and historical data query, providing users with a convenient and efficient data interaction experience.

[0168] The visualization analysis report is one of the core deliverables of the results output layer. Among them, the real-time location trajectory map of the supervisors displays the movement path of the supervisors on the construction site in an intuitive graphical way. Different colors are used to clearly mark compliant and missed locations. Compliant locations are usually displayed in green, giving people an intuitive compliance reminder; missed locations are marked in red to help managers quickly find potential problems. At the same time, the trajectory map also marks the dwell time at each location in detail, so that managers can understand at a glance the working time and dwell time of the supervisors at each location, and judge whether their work is comprehensive and in place.

[0169] The complete semantic behavior flow list details every action of the supervisor during the work process, clearly marking violations and compliant actions with different identifiers. Violations are usually marked with special symbols or eye-catching colors to attract the attention of managers; compliant actions are displayed in a normal format. In addition, the time nodes of violations and screenshots of abnormal data are also attached. The time nodes help managers quickly locate the moment when the violation occurred, and the screenshots of abnormal data intuitively show the specific data manifestations of the violations, such as abnormal movement amplitude, pressure value not meeting the standard, etc., providing detailed information for problem analysis and rectification.

[0170] In terms of compliance statistics, the output layer will accurately calculate the compliance rate, which is the ratio of the number of compliant actions to the total number of actions. This indicator can intuitively reflect the compliance level of the supervisor's behavior. At the same time, it will also count key data such as the number of times the location was missed, the number of times the sequence was incorrect, and the number of times the quality was substandard. These data show the problems in the supervisor's work from different perspectives and provide data support for managers to formulate targeted rectification measures.

[0171] For violations, the output layer will also propose specific and feasible rectification suggestions. These suggestions are based on in-depth analysis of the violations and industry experience, such as strengthening training, arranging special training courses for supervisors to improve their professional skills and operational level in certain operational irregularities or knowledge deficiencies; and optimizing work processes, redesigning and optimizing work processes to improve work efficiency and compliance for violations caused by unreasonable work processes.

[0172] In addition to visual analysis reports, the results output layer also supports data export functionality. Users can export location data, behavior flow data, and compliance results to common data formats such as Excel and CSV, facilitating further data analysis and processing. The anomaly warning push function ensures that managers can promptly understand any abnormal situations in the work of supervisors. When the system detects anomalies such as location omissions, violation of action rules, or substandard quality, it will promptly push notifications to relevant managers via SMS and system notifications, enabling them to take swift action. The historical data query function allows users to query past location data, behavior flow data, and compliance results, facilitating historical data analysis and comparison, summarizing lessons learned, and providing reference and guidance for future work. Through these functions in the results output layer, effective data display and interaction are achieved, providing support for managers' decision-making and management.

[0173] The same or similar labels correspond to the same or similar parts;

[0174] The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0175] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. For those skilled in the art, other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all implementation methods here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A method for on-site location and behavioral compliance analysis of supervisors based on the fusion of UWB and IMU, characterized in that, The method includes: The supervisors are equipped with multimodal wearable sensing devices, which include a smart safety helmet node, a waist main control positioning node, dual wrist sensing nodes and finger cot tactile nodes, to collect the supervisors' UWB positioning signals, IMU motion data, surface electromyography signals and tactile pressure data. A tightly coupled algorithm using error state Kalman filtering is employed to fuse UWB positioning signals and IMU motion data. A lightweight NLOS recognition network is introduced, and the reliability of observations is judged by the CIR characteristics of UWB signals and the consistency of IMU motion as inputs. The filter noise covariance matrix is ​​dynamically adjusted to achieve robust positioning and attitude estimation. Spatial posture features, hand motor unit features, and head-gaze features are extracted from the collected data to construct a micro-motion feature vector. Spatial posture features include body orientation and movement state, hand motor unit features include IMU trajectory template matching results, electromyographic activation patterns, and tactile pressure curves, and head-gaze features are the global head orientation vector. The semantic location labels mapped from micro-action feature vectors to UWB localization are input into the TP-LSTM-CRF model. The TP-LSTM captures short, medium, and long-term action dependencies, and the CRF layer optimizes the logicality of the behavior sequence, outputting a time-location-action semantic behavior flow with timestamps. A digital twin model of standard operating procedures is constructed for key supervision processes, defining the sequence of necessary location points, the order of standard actions at each location, and the threshold values ​​of key action parameters. The semantic behavior flow is dynamically matched with the digital twin model to detect whether there are any omissions in location, missing actions, or incorrect sequences. Based on the action amplitude, duration, and tactile pressure data, the model parameter thresholds are compared to infer whether the operation quality meets the standards. By integrating location results, behavior flow matching results, and quality inference conclusions, a preliminary compliance judgment is generated, marking the time, location, and abnormal data of violations. The above information is summarized to form a visual analysis report, including location trajectory map, compliance rate statistics, and rectification suggestions, thus completing the on-site location and behavior compliance analysis of supervisors.

2. The method according to claim 1, characterized in that, The configuration of the multimodal wearable sensing device includes: The smart safety helmet node integrates a dual-antenna UWB module and a 9-axis IMU. The dual-antenna UWB module measures the head orientation by arriving phase difference, and the 9-axis IMU identifies the head posture. The waist-mounted master positioning node includes a high-precision IMU, a UWB anchor communication module, and an edge computing unit. The high-precision IMU serves as the core of PDR to perceive body movement, while the edge computing unit runs a lightweight fusion algorithm and a basic behavior recognition program. The dual wrist sensing nodes are equipped with a 9-axis IMU and a surface electromyography (EMG) sensor. The 9-axis IMU uses high-frequency sampling to capture hand movement trajectories, while the surface EMG sensor monitors forearm muscle electrical signals to distinguish force application patterns. The finger-type tactile nodes are worn on the thumb and index finger and include miniature pressure sensors and bending sensors to measure gripping pressure and finger joint bending angle, respectively.

3. The method according to claim 1, characterized in that, The process of data fusion using the tightly coupled algorithm includes: Position, velocity, attitude, and IMU zero bias are set as the state variables of the error state Kalman filter, and the raw arrival time of UWB and IMU acceleration and angular velocity are set as the observations. The NLOS identification network analyzes the consistency between the CIR characteristic waveform of the UWB signal and the IMU motion online, and outputs the reliability level of the UWB observations. The noise covariance matrix of UWB observations in the filter is dynamically adjusted according to the reliability level. High reliability observations correspond to low noise covariance, and low reliability observations correspond to high noise covariance, thereby improving positioning stability in complex environments.

4. The method according to claim 1, characterized in that, The construction of the micro-action feature vector includes: Spatial attitude features are calculated from waist IMU data and UWB positioning results. The body orientation angle is obtained through attitude calculation, and the static or moving state is determined by combining the motion speed. In the hand motion unit features, IMU data is used to complete gesture trajectory template matching through the DTW algorithm to extract motion amplitude spectrum energy; Surface electromyography (EMG) signals were used to extract muscle activation levels, activation duration, and multi-channel signal synergistic patterns. Tactile data was used to extract the curve of peak pressure value versus pressure duration. The head-gaze feature is obtained by fusing the UWB head orientation data of the smart helmet node with the IMU attitude data, and converting it into a head orientation vector in the global coordinate system to infer the direction of the gaze focus.

5. The method according to claim 1, characterized in that, The operation of the TP-LSTM-CRF model includes: The model input layer synchronously receives micro-motion feature vectors and semantic location labels. The semantic location labels are generated by UWB positioning coordinate mapping and contain scene information. The TP-LSTM layer captures the dependencies of short-term actions, medium-term action sequences, and long-term workflows through multi-scale time windows. The CRF output layer imposes global constraints on the action sequences identified by the model, eliminates logically contradictory sequences, and ensures the time rationality of the behavior flow. The model output format is a correspondence between time interval, semantic location, and executed action.

6. The method according to claim 1, characterized in that, The construction of the digital twin model of the standard operating procedure includes: Based on construction drawings, safety regulations, and work instructions, a three-dimensional digital twin scene is built in the cloud, covering the entire process of key procedures; Define the sequence of necessary locations for each process, and mark the spatial coordinate range and minimum dwell time requirement for each location; Specify the standard sequence of actions to be performed at each location point and their order, and set parameter thresholds for key actions.

7. The method according to claim 1, characterized in that, The dynamic matching of the semantic behavior flow with the digital twin model includes: Compare the location information in the semantic behavior flow with the sequence of necessary location points in the model. If any location point is missing and the dwell time requirement is not met, it is marked as a location omission. Compare the action sequence of the semantic behavior flow with the model's standard action sequence. If there are missing actions or the order does not conform to the standard, it is marked as an action violation. The deviation between the dwell time and action execution time at each position in the statistical semantic behavior flow and the model's predicted time is counted. When the deviation exceeds a preset threshold, it is marked as an anomaly in time consumption.

8. The method according to claim 1, characterized in that, The inference of operational quality includes: Extract the amplitude, duration, and finger tactile pressure data of actions from the semantic behavior flow to form a quality analysis dataset; The dataset is compared with the threshold values ​​of key motion parameters in the digital twin model. If the motion amplitude is within the threshold range, the pressure value meets the standard, and the duration meets the requirements, the operation quality is deemed qualified. If the data exceeds the threshold range, it is marked as substandard and the specific abnormal data is recorded.

9. The method according to claim 1, characterized in that, The generation of the compliance analysis report includes: The report includes a real-time location tracking map of the supervisors, with different colors used to mark compliant and missed locations, and the time spent at each location. Present a complete list of semantic behavior flows, using different identifiers to mark violations and compliant actions, and attaching time nodes and screenshots of abnormal data for the violations; Calculate the compliance rate, which is the ratio of compliant actions to the total number of actions. Also, count the number of times the position was missed, the number of times the sequence was incorrect, and the number of times the quality was substandard. Suggestions for rectification were put forward for the violations.

10. A system for analyzing the on-site location and behavioral compliance of supervisors based on the fusion of UWB and IMU for implementing the method described in any one of claims 1-9, characterized in that, The system includes: The multimodal sensing layer consists of a smart helmet node, a waist main control positioning node, dual wrist sensing nodes, and finger-cot tactile nodes, and is responsible for collecting UWB, IMU, surface electromyography, and tactile data. The edge processing layer, deployed at the 5G / MEC gateway on the construction site, runs a tightly coupled fusion algorithm and an NLOS recognition network, and outputs robust localization results and micro-motion feature vectors. The model computation layer includes TP-LSTM-CRF model units and digital twin modeling units. The former outputs semantic behavior flow, while the latter constructs a digital model of standard operating procedures. The compliance analysis layer includes a dynamic rule matching module and a quality inference module. The dynamic rule matching module detects location / action violations, and the quality inference module judges the quality of operations and generates compliance results. The results output layer aggregates location data, behavior flow, and compliance results to generate visual analysis reports, supporting data export, anomaly alert push notifications, and historical data queries.

Images