A construction worker safety state monitoring method and system based on multi-modal data fusion

Abstract

A kind of building worker safety state monitoring method and system based on multi-modal data fusion, its method includes: by bluetooth low power consumption protocol, wireless connection is established with inertial measurement unit, receives and parses multi-channel motion data, obtains multidimensional motion state vector;Three constraints fall detection model is constructed, from input layer starts in turn connection acceleration impact detection layer, posture anomaly detection layer, time window constraint layer, logic fusion layer and output layer, to combined acceleration impact, head posture anomaly and impact duration are jointly judged;Synchronous acquisition hardware tilt sensor's level signal and software tilt determination result carry out logic and operation, realize dual sensor redundancy;When fall event is judged as true, trigger video lockout interactive process, freeze video frame, superimpose warning information and continue sound and light alarm, until manual confirmation is released lockout.The present application is suitable for worker safety protection, fall detection, wearing supervision and operation quality evaluation under the scene of building site painting operation, with the characteristics of high detection accuracy, good system reliability, man-machine interaction friendly, quality evaluation objectification etc.

Description

Technical Field

[0001] This invention relates to the field of construction site safety management and smart wearable device technology, specifically to a method and system for monitoring the safety status of construction workers based on multimodal data fusion. It collects data from inertial measurement units via Bluetooth Low Energy protocol, constructs a triple-constraint fall detection model, and combines a dual-sensor redundancy architecture with a video blocking interaction mechanism to achieve high-precision detection and emergency response to construction worker fall events. It also integrates an image quality assessment function for painting work surfaces, making it particularly suitable for worker safety protection and work quality monitoring in painting construction scenarios. Background Technology

[0002] In the construction industry, especially in indoor and outdoor painting operations, worker safety and work efficiency monitoring have always been challenging and painful management issues. If a worker falls while working at heights or on scaffolding, failure to detect it promptly can lead to secondary injuries or even more serious consequences. While some smart safety helmet products have emerged, they generally suffer from the following shortcomings: (1) High false alarm rate of fall detection: Most products use a single acceleration threshold judgment method, which judges a fall as when the combined acceleration exceeds a certain fixed value. However, in actual applications, workers’ daily actions such as bending over quickly, squatting and standing up, and carrying heavy objects may generate similar acceleration impacts, resulting in a high false alarm rate, which seriously affects the practicality and reliability of the system.

[0003] (2) Risk of single-point failure of sensor: Many products rely solely on software algorithms for attitude detection. Once the inertial measurement unit experiences data drift or communication interruption, the entire safety monitoring function immediately fails, lacking hardware-level backup protection.

[0004] (3) Lack of human-computer interaction confirmation mechanism: Most existing fall alarm systems are one-way alarm modes, that is, they issue sound and light prompts after detecting an abnormality, but cannot force on-site personnel or managers to confirm and handle the situation. In unattended scenarios, this may lead to the risk of alarms being ignored.

[0005] (4) Lack of painting quality assessment methods: Traditional painting construction relies on manual visual inspection and lacks objective image-based quality assessment methods. Furthermore, due to the complex lighting conditions at the construction site, the collected images generally have central brightness deviation and vignetting effect, which directly affects the accuracy of the assessment. Summary of the Invention

[0006] The present invention aims to overcome the defects and deficiencies of the prior art and provide a method and system for monitoring the safety status of construction workers based on multimodal data fusion.

[0007] This invention significantly improves the accuracy of fall detection, enhances system reliability, and improves the human-computer interaction experience by constructing a triple-constraint fall detection model, a dual-sensor redundancy architecture, a video blocking interaction mechanism, and a subsystem for evaluating the image quality of the painting work surface. At the same time, it enables an objective evaluation of the quality of painting construction.

[0008] The technical solution of the present invention is as follows: A method for monitoring the safety status of construction workers based on multimodal data fusion includes the following steps: S1. Establish a wireless connection with the inertial measurement unit via Bluetooth Low Energy protocol, receive and parse the multi-channel motion data output by it, and obtain a multi-dimensional motion state vector including acceleration, angular velocity, attitude angle, position coordinates and motion activity type. S2. Construct a triple-constraint fall detection model to jointly determine the result of the combined acceleration impact, abnormal head posture, and impact duration, and output the fall event determination result. S3. Synchronously acquire the level signal of the hardware tilt sensor, perform a logical AND operation with the software tilt judgment result calculated based on the multi-dimensional motion state vector, and output a redundant tilt alarm signal; S4. When the fall event determination result is true, the video blocking interaction process is triggered, the current video frame is frozen and a warning message is superimposed, and the sound and light alarm output is activated until the blocking is lifted after receiving manual confirmation. S5. Preprocess and assess the quality of the painted surface image, and output the quality assessment results including surface flatness, color uniformity, defect density, and comprehensive score.

[0009] The structure of the triple-constraint fall detection model described in step S2 is as follows: starting from the input layer, it is sequentially connected to the acceleration impact detection layer, the posture anomaly detection layer, the time window constraint layer, the logic fusion layer, and the output layer. The input layer is used to receive the triaxial acceleration components and Euler angle components in the multi-dimensional motion state vector and distribute them to the subsequent detection layers. The acceleration impact detection layer is used to calculate the vector composite acceleration of the three-axis acceleration, compare the composite acceleration with a preset acceleration threshold, and output a first-level warning flag and record the impact start time when the composite acceleration exceeds the preset acceleration threshold; otherwise, the first-level warning flag is cleared. The posture anomaly detection layer is used to monitor the absolute value of the head's Euler angles. When the absolute value of any Euler angle exceeds a preset angle threshold, a second-level warning flag is output; otherwise, the second-level warning flag is cleared. The time window constraint layer is used to calculate the duration difference between the current time and the impact start time, and outputs an instantaneous impact flag when the duration difference is less than or equal to a preset time window. The logic fusion layer is used to perform a logical AND operation on the instantaneous impact flag and the second-level warning flag, and outputs the fall event judgment result only when both are true. The output layer is used to output the fall event determination result, and automatically release the fall state when the resultant acceleration returns to the normal gravity range and the absolute value of the head Euler angle returns to the normal range.

[0010] The second-level warning flag output by the attitude anomaly detection layer is logically ANDed with the level signal of the hardware tilt sensor to achieve dual sensor redundancy for tilt detection, so that a fault in any detection path will not cause false alarms or missed alarms.

[0011] The video blocking interaction process described in step S4 includes: S41. Capture and save the current video frame as a lock frame the instant the fall event is determined to be real, and stop the video refresh loop; S42. Overlay accident warning information and manual confirmation prompts onto the locked frame; S43. Continuously output audible and visual alarm signals until user confirmation is detected; S44. The video blocking interaction process simultaneously realizes the dual functions of accident scene evidence collection and manual safety management.

[0012] Among them, step S42, which involves overlaying the accident warning information and manual confirmation prompt onto the locked frame, includes: S421. Overlay at least three layers of text information onto the locked frame, namely, accident type identifier, event description information, and manual unlocking operation instructions; S422. The audible and visual alarm signal includes the coordinated output of a buzzer audible alarm and a dual-color LED flashing alarm, and is maintained continuously until the video blocking process is lifted.

[0013] Step S5, which involves preprocessing the image of the painting work surface, includes: S45. Establish a radial distance model with the image center as the origin, and perform center brightness deviation elimination and vignetting radial brightness compensation on the image based on the radial distance model; S46. Perform edge-preserving denoising on the compensated image and overlay a high-frequency detail mask to enhance local contrast, resulting in a preprocessed image.

[0014] Furthermore, the radial brightness compensation for dark corners in step S45 adopts an adaptive nonlinear gain curve that is inversely proportional to the pixel brightness, so that pixels in dark areas get higher gain and pixels in bright areas get lower gain, avoiding highlight overflow and noise amplification.

[0015] Wearing status monitoring includes: S51. Wearing pressure data is collected by a weighing sensor, and the wearing pressure data is smoothed by an exponential moving average filtering algorithm; S52. Compare the smoothed wearing pressure data with the preset effective wearing pressure range, and output the wearing status determination result.

[0016] The method of the present invention further includes: performing drift suppression processing on the position coordinates, and suppressing the position drift error caused by acceleration integral accumulation through a dual mechanism of position jump threshold filtering and zero-point reset detection; The multi-dimensional motion state vectors are batch recorded and persisted. The multi-dimensional motion state vectors are temporarily stored in a circular buffer of fixed capacity. When the circular buffer reaches the preset capacity, a batch write operation is triggered. The batch write operations and multi-threaded concurrent access are protected by mutex locks to ensure data consistency.

[0017] Another aspect of the present invention relates to a construction worker safety status monitoring system based on multimodal data fusion, which performs the method of the present invention, the system comprising: The inertial measurement unit data acquisition module is used to acquire and parse multi-channel motion data via Bluetooth Low Energy protocol; The triple-constraint fall detection module is used to jointly determine the combined acceleration impact, abnormal head posture, and impact duration. The dual-redundant tilt determination module is used to perform a logical AND operation between the level signal of the hardware tilt sensor and the tilt determination result of the software. The video blocking interaction module is used to freeze video frames, overlay warning information, and enter a blocking state when a fall event occurs; Wearing status monitoring module, used to determine the wearing status based on weighing sensor data; The painting quality assessment module is used to preprocess and score the painting work surface images in multiple dimensions. The data communication and recording module is used to upload, cache, and persist sensor data.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. Significantly improved fall detection accuracy: Through the layered architecture design of the triple-constraint fall detection model, acceleration impact detection, posture anomaly detection, and time window constraints are jointly determined through a logic fusion layer, effectively filtering out false positive interference caused by daily actions and significantly reducing the false alarm rate.

[0019] 2. Significantly enhanced system reliability: The introduction of a dual-sensor redundancy architecture elevates the availability of critical safety functions from a single point of failure mode to a fault-tolerant mode. Even if either the inertial measurement unit or the tilt sensor fails, the system can still degrade to provide basic safety assurance.

[0020] 3. Revolutionary improvement in human-computer interaction experience: The video blocking interaction mechanism has created a new human-computer interaction model in the field of safety helmets, realizing the dual functions of accident scene evidence collection and manual safety management, and avoiding the safety hazards that automatic recovery may bring.

[0021] 4. Objectification of painting quality assessment: The radial distance model is used to eliminate the center brightness deviation and dark corner effect, and a weighted fusion score is performed based on multi-dimensional indicators on the basis of preprocessing, so as to realize the objectification and visualization assessment of painting construction quality. Attached Figure Description

[0022] Figure 1 This is a block diagram of the overall architecture of the system of the present invention; Figure 2 This is a schematic diagram of the shelf structure of the triple-constraint fall detection model of the present invention; Figure 3 This is a logical schematic diagram of the dual-sensor redundancy architecture of the present invention; Figure 4 This is a state transition diagram of the video blocking interaction mechanism of the present invention; Figure 5 This is a schematic diagram of the multi-channel motion data bit field mask parsing process of the present invention; Figure 6 This is a diagram of the data processing pipeline architecture of the present invention; Figure 7 This is a schematic diagram of the structure of the smart safety helmet of the present invention; Figure 8 This is a schematic cross-sectional view of the smart safety helmet of the present invention. Detailed Implementation

[0023] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0024] Example 1

[0025] This embodiment relates to the hardware of a construction worker safety status monitoring system that implements the multimodal data fusion-based construction worker safety status monitoring method of the present invention, and its composition is as follows: (1) Main control unit: The core processor is a Raspberry Pi single-board computer running the Ubuntu operating system. It is equipped with an ARM Cortex-A72 quad-core processor with a main frequency of 1.5GHz and 4GB of memory, which is sufficient to meet the needs of real-time data processing and multi-task scheduling.

[0026] (2) Inertial Measurement Unit: A six-axis / nine-axis inertial measurement unit supporting the BLE 4.2 protocol is selected, with a built-in accelerometer range of ±8g, a gyroscope range of ±2000dps, a magnetometer range of ±4800μT, and a sampling rate of up to 100Hz. The sensor is wirelessly connected to the main control unit via Bluetooth, with a communication distance of ≥10 meters.

[0027] (3) Weighing sensor: The strain gauge load sensor driven by the HX711 dedicated AD conversion module is installed at the position where the top of the safety helmet liner contacts the top of the person's head. The range is 0-50kg and the accuracy is ±5g. It is used to detect whether the safety helmet is worn correctly.

[0028] (4) Hardware tilt sensor: An independent mechanical or MEMS tilt switch sensor is used, which is installed on the side of the safety helmet and directly connected to the main control unit through the GPIO interface as a hardware-level tilt detection backup.

[0029] (5) Camera: A high-definition camera module supporting GStreamer pipeline is used, with a resolution of 480×320 and a frame rate of 30fps. It is installed at the front edge of the safety helmet, with the lens facing the work surface.

[0030] (6) Alarm output module: includes a dual-color LED (red + green) and a piezoelectric buzzer, both controlled by GPIO.

[0031] (7) Power module: It adopts a 5000mAh rechargeable lithium-ion battery pack, supports 5V / 2A output, and has a battery life of ≥12 hours.

[0032] Example 2

[0033] This embodiment relates to a method for monitoring the safety status of construction workers based on multimodal data fusion, including the following steps: S1. Establish a wireless connection with the inertial measurement unit via Bluetooth Low Energy protocol, receive and parse the multi-channel motion data output by it, and obtain a multi-dimensional motion state vector including acceleration, angular velocity, attitude angle, position coordinates and motion activity type. S2. Construct a triple-constraint fall detection model to jointly determine the result of the combined acceleration impact, abnormal head posture, and impact duration, and output the fall event determination result. S3. Synchronously acquire the level signal of the hardware tilt sensor, perform a logical AND operation with the software tilt judgment result calculated based on the multi-dimensional motion state vector, and output a redundant tilt alarm signal; S4. When the fall event determination result is true, the video blocking interaction process is triggered, the current video frame is frozen and a warning message is superimposed, and the sound and light alarm output is activated until the blocking is lifted after receiving manual confirmation. S5. Preprocess and assess the quality of the painted surface image, and output the quality assessment results including surface flatness, color uniformity, defect density, and comprehensive score.

[0034] Step S1 is the process of establishing a wireless connection with the inertial measurement unit via Bluetooth Low Energy protocol and parsing multi-channel motion data.

[0035] The master control unit establishes a Common Attribute Profile (GATT) connection with the inertial measurement unit (IMU) via Bluetooth Low Energy protocol. After the connection is established, the master control unit sends an initialization command sequence to the IMU's write feature values ​​to configure the sensor's data output mode, including enabling data channels such as acceleration, angular velocity, magnetometer, quaternion, Euler angle, position coordinates, motion activity recognition, and step count. After configuration, the IMU periodically pushes sensor data to the master control unit via notification feature values.

[0036] The data frames received by the main control unit are identified by a preset start byte 0x11, followed by a 2-byte control word. This control word is a 16-bit unsigned integer, with each bit mapped to a different sensor data channel group: bit 0 indicates the presence of triaxial acceleration data, bit 1 indicates the presence of triaxial angular velocity data, bit 2 indicates the presence of triaxial magnetometer data, bit 4 indicates the presence of quaternion attitude data, bit 6 indicates the presence of triaxial Euler angle data, bit 7 indicates the presence of triaxial raw position coordinate data, and bit 8 indicates the presence of motion activity probability flags and step count data.

[0037] The main control unit performs bitwise operations to parse the control word, dynamically determines the valid data channels contained in the current data frame based on the set bits, and sequentially extracts the original analog-to-digital conversion data according to the preset offsets corresponding to each channel. The extracted original data is multiplied by the physical quantity conversion coefficients of the corresponding channels to obtain a multi-dimensional motion state vector with physical meaning, including triaxial acceleration, triaxial angular velocity, triaxial magnetometer reading, quaternion attitude, Euler angles, original position coordinates, probabilities of four types of motion activities (walking / running / cycling / driving), and step count.

[0038] In addition, to prevent high-frequency notifications from causing processing congestion, the main control unit has a built-in 50-millisecond minimum interval filtering mechanism. When the interval between two notifications is less than 50 milliseconds, the data of the next frame is discarded to ensure the real-time performance and stability of data processing.

[0039] The structure and workflow of the triple-constraint fall detection model in step S2. (For example...) Figure 2 As shown, the structure of the triple-constraint fall detection model is as follows: starting from the input layer, it is sequentially connected to the acceleration impact detection layer, the posture anomaly detection layer, the time window constraint layer, the logic fusion layer, and the output layer.

[0040] (1) Input layer

[0041] The input layer receives the triaxial acceleration components (ax, ay, az) and Euler angle components (roll, pitch) from the multi-dimensional motion state vector obtained in step S1, and distributes them to subsequent detection layers. The triaxial acceleration components come from the acceleration channel of the inertial measurement unit, and the Euler angle components come from the Euler angle channel, reflecting the linear motion impact of the head and the attitude deflection state, respectively.

[0042] (2) Acceleration impact detection layer

[0043] The acceleration impact detection layer calculates the vector composite acceleration a = √(ax² + ay² + az²) of the three-axis accelerations and compares this composite acceleration with a preset acceleration threshold. In this embodiment, the preset acceleration threshold is set to 2.5g. This threshold is set based on the principles of human kinematics: the acceleration of normal human movement usually does not exceed 2g, while the impact acceleration at the moment of a fall can often reach more than 3g. When the composite acceleration exceeds 2.5g, the layer outputs a first-level warning flag (set to true) and records the current timestamp as the impact start time; when the composite acceleration is less than or equal to 2.5g, the first-level warning flag and the impact start time are cleared.

[0044] (3) Attitude Anomaly Detection Layer

[0045] The posture anomaly detection layer monitors the absolute values ​​of the head's Euler angles, including roll and pitch. When the absolute value of any Euler angle exceeds a preset angle threshold, a second-level warning flag is output. In this embodiment, the preset angle threshold is set to 60°, which is determined based on ergonomic analysis: when standing normally or during slight activity, the head tilt angle generally does not exceed 30°, while during a fall, the head usually tilts violently to a near-horizontal or even inverted state. When the absolute value of any Euler angle exceeds 60°, this layer outputs a second-level warning flag (set to true); otherwise, the flag is cleared.

[0046] (4) Time window constraint layer

[0047] The time window constraint layer calculates the duration difference between the current moment and the impact start moment. When the duration difference is less than or equal to a preset time window, an instantaneous impact flag is output. In this embodiment, the preset time window is set to 1.0 second. The design principle of this layer is that a real fall event is a continuous action process, with acceleration impact and posture change occurring almost simultaneously; while everyday non-fall actions (such as quickly bending over) may generate large accelerations, their posture changes are usually gradual and controlled, and will not simultaneously meet both conditions in a very short time. Through time window constraints, such false positive interference can be effectively filtered out.

[0048] (5) Logical Fusion Layer

[0049] The logic fusion layer performs a logical AND operation on the instantaneous impact flag and the second-level warning flag, outputting a fall event determination result as true only when both are true simultaneously. This design ensures that a fall is determined only when acceleration impact and abnormal posture occur simultaneously within the time window, eliminating the possibility of misjudgment based on a single indicator from a logical perspective.

[0050] (6) Output layer

[0051] The output layer outputs the fall event determination result and has a self-recovery mechanism. When the resultant acceleration returns to the normal gravity range (between 0.8g and 1.2g) and the absolute value of the head Euler angle is less than 45°, the fall state is automatically deactivated and normal monitoring resumes without manual intervention. This self-recovery mechanism can handle situations where workers get up on their own after a fall.

[0052] The working principle of the dual-sensor redundancy architecture in step S3. For example... Figure 3 As shown, the architecture deploys two independent tilt detection paths.

[0053] The first detection path is a software algorithm path: it utilizes high-precision nine-axis data collected by the inertial measurement unit, and uses the second-level warning flag calculated by the attitude anomaly detection layer in step S2 as the software tilt determination result. The advantages of this path are high accuracy, strong configurability, and the ability to provide rich intermediate data for analysis.

[0054] The second detection path is a hardware sensing path: an independent tilt sensor is installed on the helmet itself, and this sensor is directly connected to the main control chip via a GPIO interface. When the tilt angle of the helmet exceeds the mechanical threshold, the internal contacts of the sensor close, outputting a high-level signal; otherwise, it outputs a low-level signal. The main control chip reads the level state of this GPIO pin to obtain the hardware tilt alarm signal. This path operates completely independently of the inertial measurement unit and software algorithm; even if the inertial measurement unit malfunctions or the software crashes, the hardware sensor can still work independently.

[0055] The main control processing module performs a logical AND operation between the second-level warning flag output from the attitude anomaly detection layer and the level signal from the hardware tilt sensor. Only when both signals are true simultaneously is the final tilt alarm signal output. This design offers two major advantages: first, it prevents false alarms, as any occasional error signal from any single channel will not trigger an alarm independently; second, it prevents missed alarms, as even if one channel fails, the other can still operate independently. In the comprehensive judgment of the actual system, the main control processing module calculates the final alarm conditions as follows: the wearing status is "worn" (weighing sensor reading is within the valid range), the acceleration exceeds the limit (the combined acceleration exceeds the preset threshold), and (the hardware tilt sensor detects tilt or the inertial measurement unit detects tilt). This comprehensive judgment logic superimposes the wearing status constraint on top of the fall detection model, further reducing the false alarm rate.

[0056] The workflow of the video blocking interaction mechanism in step S4. For example... Figure 4 As shown, when the triple-constraint fall detection model determines that a fall event has occurred and the comprehensive alarm conditions are met, the system executes the following steps: S41. Frame Freeze and Lock Frame Capture The moment a fall is confirmed, the main control module immediately captures the latest frame from the current camera, calls `frame.copy()` to copy and save it as a locked frame, and simultaneously stops the normal video refresh loop. This frozen frame will then be displayed on the screen indefinitely, without further updates. The locked frame preserves visual evidence of the moment the fall occurred, which can be used for post-incident analysis and liability determination.

[0057] S42. Warning message overlay

[0058] Based on the locked frame, the system overlays at least three layers of text information using a text rendering function: the first layer is the accident type identifier, displayed in bold red "ALERT"; the second layer is the event description information, displayed in red "FALLDETECTED"; and the third layer is the manual unlocking operation guide, displayed in yellow "CLICK TO UNLOCK". The positions of these texts are carefully designed to ensure they are clearly visible against any background.

[0059] S43. Continuous output of audible and visual alarm.

[0060] At the same time, the buzzer starts to sound (outputting a LOW level in low-level trigger mode), and the dual-color LED enters a red-green alternating flashing mode (the flashing period is controlled by the LED_BLINK_PERIOD parameter, with a default of 0.3 seconds). The audible and visual alarm will continue unaffected by subsequent changes in sensor data until it is manually deactivated.

[0061] S44. Video Blocking Maintenance and Manual Unblocking

[0062] Once the system enters video blocking mode, the internal state variables `video_locked`, `is_alert`, and `is_falling` are set to `True`. The only way to unlock the video blocking is by detecting a user's left mouse click event. Upon detecting a click, the system performs the following actions: resets all three state variables (`video_locked`, `is_alert`, and `is_falling`) to `False`; clears the `locked_frame` reference to release memory; stops the buzzer; restores the LEDs to normal; and restarts the normal video refresh cycle. This mechanism simultaneously achieves the dual functions of accident scene evidence collection and manual safety management, avoiding potential safety hazards associated with automatic recovery.

[0063] The specific implementation method of wearing status monitoring is as follows: The main control unit communicates with the weighing sensor through the serial port, sends a weight request command every 200 milliseconds, and reads the weight value in the returned data frame.

[0064] To eliminate reading fluctuations in the weighing sensor, an exponential moving average (EMA) filtering algorithm is used to smooth the raw wearing pressure data. The filtering formula is: W_new = 0.75 × W_old + 0.25 × W_raw, where W_raw is the latest raw reading, W_old is the filtered output value at the previous moment, and W_new is the filtered output value at the current moment. A smoothing factor of 0.25 effectively suppresses reading fluctuations within a ±10g range while maintaining response speed.

[0065] The filtered wearing pressure data is compared with a preset effective wearing pressure range. In this embodiment, the effective wearing pressure range is set to 270g to 330g, which is calibrated based on the actual pressure distribution between the helmet liner and the head. When the filtered output value is within this range, it is determined that the helmet is being worn, and the green light remains on. When the filtered output value is below the lower limit of this range, it is determined that the helmet is not being worn, and the red light illuminates and a buzzer is triggered as a reminder.

[0066] The preprocessing process for images of the painting work surface is as follows: Due to the complex lighting conditions at the construction site, the acquired images generally exhibit uneven brightness distribution, with the center bright and the edges dark, directly affecting the accuracy of subsequent quality assessment. This invention employs the following preprocessing steps: S45. Radial Distance Model Establishment and Brightness Deviation Elimination First, the acquired raw image is converted to grayscale space, and a two-dimensional radial distance matrix is ​​constructed with the image center as the origin and normalized to the range [-1,1]. Based on this radial distance matrix, a center brightness deviation compensation factor is calculated using a Gaussian decay function, and the grayscale image is divided by this compensation factor to eliminate the center halo effect.

[0067] Subsequently, a vignetting radial brightness compensation model is established on the radial distance matrix. The base gain is calculated with the transition radius ratio as the boundary, and an adaptive nonlinear gain curve is established based on the ratio of pixel brightness to maximum brightness. This gain curve is inversely proportional to pixel brightness; that is, pixels in dark areas receive higher gain, and pixels in bright areas receive lower gain, thus brightening the vignetting edges while avoiding highlight clipping and noise amplification. The image after eliminating the central halo effect is multiplied by the adaptive nonlinear gain curve to obtain a brightness-uniformed image.

[0068] S46. Edge Preservation Denoising and Local Contrast Enhancement

[0069] Bilateral filtering is performed on the brightness-uniformed image to preserve edge information while denoising. A high-frequency detail mask is then extracted (obtained by the difference between the original image and the Gaussian blur image), and this detail mask is superimposed back onto the filtering result with enhanced weights to obtain a preprocessed image with enhanced local contrast. This preprocessed image effectively preserves key features such as wall texture and cracks, providing high-quality input for subsequent quality assessment.

[0070] Based on the preprocessed image, this invention evaluates the quality of the painted surface from four dimensions: (1) Surface smoothness: The horizontal and vertical gradient magnitudes of the image are calculated based on the Sobel operator, and the average gradient magnitude is taken as the surface smoothness score. The higher the average gradient magnitude, the richer the wall texture and the better the smoothness.

[0071] (2) Color uniformity: Convert the image to the LAB color space and calculate the sum of the standard deviations of the A and B channels. The smaller the standard deviation, the more uniform the color distribution, and the higher the color uniformity score.

[0072] (3) Defect density: Edge pixels in the image are extracted using Canny edge detection, and the proportion of edge pixels to total pixels is calculated. The lower the edge proportion, the fewer the wall defects, and the higher the defect density score.

[0073] (4) Comprehensive scoring: The scores of the above three dimensions are weighted and fused according to the preset weights. In this embodiment, the weights are allocated as follows: surface flatness 0.3, color uniformity 0.25, defect density 0.25, and overall visual consistency 0.2. The comprehensive score is further mapped to the A / B / C / D grade system: 90 points and above is grade A, 80-89 points is grade B, 70-79 points is grade C, and below 70 points is grade D.

[0074] The process of suppressing position coordinate drift is as follows: The position data of the inertial measurement unit is obtained by integrating the acceleration twice, which inevitably results in accumulated drift error. This invention employs two strategies to jointly suppress drift: (1) Position jump threshold filtering: Calculate the difference vector of the original position coordinates between two adjacent sampling times. If the absolute value of any component of the difference vector exceeds the preset position jump threshold (set to 1.5 meters in this embodiment), it is determined to be an abnormal position jump, and the current position update is discarded, while the cumulative position of the previous time remains unchanged. This strategy can effectively filter out sudden jumps caused by sensor noise or communication interference.

[0075] (2) Zero-point reset detection: When the absolute value of each component of the original position coordinate at the current sampling time is less than the preset zero-point reset threshold (set to 0.02 meters in this embodiment), and the absolute value of at least one component of the original position coordinate at the previous time is greater than the threshold, it is determined that the inertial measurement unit has performed a position calibration reset operation, and the accumulated position is reset to zero. This strategy can identify calibration events inside the sensor and prevent the accumulated position from deviating from the actual position.

[0076] The process of batch recording and persisting multi-dimensional motion state vectors is as follows: The sampling frequency of inertial measurement unit data is high (up to 100Hz or more). If each data is immediately written to the disk file, frequent I / O operations will become a performance bottleneck.

[0077] This invention employs a strategy of first caching and then batch processing: newly arriving multi-dimensional motion state vectors are first appended to a fixed-capacity circular buffer (10 vectors in this embodiment); when the buffer reaches its preset capacity, a batch write operation is triggered, appending all data in the buffer to a CSV file at once, and then the buffer is cleared. This strategy combines what were originally N independent small-scale I / O operations into one large-scale I / O operation, significantly improving data recording throughput.

[0078] Furthermore, the system employs a multi-threaded architecture—the main thread handles UI rendering and user interaction, the inertial measurement unit callback thread receives and processes sensor data, and the data logging thread handles file writing. All data logging operations are protected by mutex locks to prevent data races caused by concurrent access from multiple threads, ensuring data consistency.

[0079] Example 3

[0080] like Figure 1 As shown, the construction worker safety status monitoring system based on multimodal data fusion of the present invention includes the following modules: (1) Inertial Measurement Unit Data Acquisition Module: Used to acquire and parse multi-channel motion data via Bluetooth Low Energy protocol, establish GATT connection, send initialization command sequence, receive variable length data frames and perform bit field mask adaptive parsing.

[0081] (2) Triple constraint fall detection module: used to jointly determine the combined acceleration impact, abnormal head posture and impact duration. It adopts a layered architecture from the input layer to the output layer and outputs the fall event determination result through the logic fusion layer.

[0082] (3) Dual-channel redundant tilt determination module: used to perform logical AND operation between the level signal of the hardware tilt sensor and the tilt determination result of the software to realize dual sensor redundancy of tilt detection.

[0083] (4) Video blocking interaction module: used to freeze video frames, overlay warning information and enter the blocking state when a fall occurs, and continuously output sound and light alarms until a manual confirmation operation is detected.

[0084] (5) Wearing status monitoring module: used to determine the wearing status based on the weighing sensor data, use the exponential moving average filtering algorithm to eliminate sensor noise, and output the wearing status through the pressure threshold range determination method.

[0085] (6) Painting quality assessment module: used to preprocess and score the painting work surface image in multiple dimensions, including radial distance model brightness compensation, bilateral filtering denoising, Sobel / LAB / Canny multi-dimensional index calculation and weighted fusion scoring.

[0086] (7) Data communication and recording module: used to realize the uploading, caching and persistence of sensor data, and adopts a ring buffer batch writing strategy and a mutex lock thread safety mechanism.

[0087] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the scope defined by the claims of the present invention.

Claims

1. A method for monitoring the safety status of construction workers based on multimodal data fusion, characterized in that, include: S1. Establish a wireless connection with the inertial measurement unit via Bluetooth Low Energy protocol, receive and parse the multi-channel motion data output by it, and obtain a multi-dimensional motion state vector including acceleration, angular velocity, attitude angle, position coordinates and motion activity type. S2. Construct a triple-constraint fall detection model to jointly determine the result of the combined acceleration impact, abnormal head posture, and impact duration, and output the fall event determination result. S3. Synchronously acquire the level signal of the hardware tilt sensor, perform a logical AND operation with the software tilt judgment result calculated based on the multi-dimensional motion state vector, and output a redundant tilt alarm signal; S4. When the fall event determination result is true, the video blocking interaction process is triggered, the current video frame is frozen and a warning message is superimposed, and the sound and light alarm output is activated until the blocking is lifted after receiving manual confirmation. S5. Preprocess and assess the quality of the painted surface image, and output the quality assessment results including surface flatness, color uniformity, defect density, and comprehensive score.

2. The method according to claim 1, characterized in that, The structure of the triple-constraint fall detection model described in step S2 is as follows: starting from the input layer, it is sequentially connected to the acceleration impact detection layer, the posture anomaly detection layer, the time window constraint layer, the logic fusion layer, and the output layer; The input layer is used to receive the triaxial acceleration components and Euler angle components in the multi-dimensional motion state vector and distribute them to the subsequent detection layers. The acceleration impact detection layer is used to calculate the vector composite acceleration of the three-axis acceleration, compare the composite acceleration with a preset acceleration threshold, and output a first-level warning flag and record the impact start time when the composite acceleration exceeds the preset acceleration threshold; otherwise, the first-level warning flag is cleared. The posture anomaly detection layer is used to monitor the absolute value of the head's Euler angles. When the absolute value of any Euler angle exceeds a preset angle threshold, a second-level warning flag is output; otherwise, the second-level warning flag is cleared. The time window constraint layer is used to calculate the duration difference between the current time and the impact start time, and outputs an instantaneous impact flag when the duration difference is less than or equal to a preset time window. The logic fusion layer is used to perform a logical AND operation on the instantaneous impact flag and the second-level warning flag, and outputs the fall event judgment result only when both are true. The output layer is used to output the fall event determination result, and automatically release the fall state when the resultant acceleration returns to the normal gravity range and the absolute value of the head Euler angle returns to the normal range.

3. The method according to claim 2, characterized in that, The second-level warning flag output by the attitude anomaly detection layer is logically ANDed with the level signal of the hardware tilt sensor to achieve dual sensor redundancy for tilt detection, so that a fault in any detection path will not cause false alarms or missed alarms.

4. The method according to claim 1, characterized in that, Step S4, the video blocking interaction process, includes: S41. Capture and save the current video frame as a lock frame the instant the fall event is determined to be real, and stop the video refresh loop; S42. Overlay accident warning information and manual confirmation prompts onto the locked frame; S43. Continuously output audible and visual alarm signals until user confirmation is detected; S44. The video blocking interaction process simultaneously realizes the dual functions of accident scene evidence collection and manual safety management.

5. The method according to claim 4, characterized in that, Step S42, which involves overlaying accident warning information and manual confirmation prompts onto the locked frame, includes: S421. Overlay at least three layers of text information onto the locked frame, namely, accident type identifier, event description information, and manual unlocking operation instructions; S422. The audible and visual alarm signal includes the coordinated output of a buzzer audible alarm and a dual-color LED flashing alarm, and is maintained continuously until the video blocking process is lifted.

6. The method according to claim 1, characterized in that, Step S5, which involves preprocessing the painting surface image, includes: S45. Establish a radial distance model with the image center as the origin, and perform center brightness deviation elimination and vignetting radial brightness compensation on the image based on the radial distance model; S46. Perform edge-preserving denoising on the compensated image and overlay a high-frequency detail mask to enhance local contrast, resulting in a preprocessed image.

7. The method according to claim 6, characterized in that, The radial brightness compensation for dark corners in step S45 adopts an adaptive nonlinear gain curve that is inversely proportional to the pixel brightness, so that pixels in dark areas get higher gain and pixels in bright areas get lower gain, avoiding highlight overflow and noise amplification.

8. The method according to claim 1, characterized in that, Wearing status monitoring, including: S51. Wearing pressure data is collected by a weighing sensor, and the wearing pressure data is smoothed by an exponential moving average filtering algorithm; S52. Compare the smoothed wearing pressure data with the preset effective wearing pressure range, and output the wearing status determination result.

9. The method according to any one of claims 1 to 8, characterized in that, Also includes: The position coordinates are subjected to drift suppression processing. The position jump threshold filtering and zero-point reset detection are used as dual mechanisms to suppress the position drift error caused by acceleration integral accumulation. The multi-dimensional motion state vectors are batch recorded and persisted. The multi-dimensional motion state vectors are temporarily stored in a circular buffer of fixed capacity. When the circular buffer reaches the preset capacity, a batch write operation is triggered. The batch write operations and multi-threaded concurrent access are protected by mutex locks to ensure data consistency.

10. A construction worker safety status monitoring system based on multimodal data fusion, characterized in that, The system performs the method according to any one of claims 1 to 9, wherein the system comprises: The inertial measurement unit data acquisition module is used to acquire and parse multi-channel motion data via Bluetooth Low Energy protocol; The triple-constraint fall detection module is used to jointly determine the combined acceleration impact, abnormal head posture, and impact duration. The dual-redundant tilt determination module is used to perform a logical AND operation between the level signal of the hardware tilt sensor and the tilt determination result of the software. The video blocking interaction module is used to freeze video frames, overlay warning information, and enter a blocking state when a fall event occurs; Wearing status monitoring module, used to determine the wearing status based on weighing sensor data; The painting quality assessment module is used to preprocess and score the painting work surface images in multiple dimensions. The data communication and recording module is used to upload, cache, and persist sensor data.

Images