Multi-objective correlation unsafe behavior analysis method and system for construction work environment

Abstract

The present application relates to the fields of construction and security technology, and provides a method and system for analyzing unsafe behaviors in a construction work environment with multiple targets associated. The method comprises: obtaining an image of the construction work environment captured in real time by a camera; performing target detection on the image of the construction work environment, detecting various types of targets and outputting the key points of each target, wherein the key points of the target include the center point of the target and the upper and lower edge endpoints of the target; generating feature fusion weights between any two targets using the key points of the target; performing feature fusion using the feature fusion weights between any two targets and the features of each target, identifying behavioral safety risks based on the fused features and outputting the risk level; the present invention improves the accuracy of dangerous behavior identification, and can effectively reduce the false alarm rate and improve the response speed.

Description

Technical Field

[0001] The present application relates to the fields of construction and security technology, and in particular to a method and system for analyzing unsafe behaviors in a construction work environment with multi-target associations. Background Art

[0002] With the acceleration of urbanization, construction site safety management faces severe challenges. Traditional safety monitoring relies on manual inspections, which suffers from low efficiency and delayed response, making it difficult to adapt to the dynamic safety management needs of modern large-scale construction sites.

[0003] Current deep learning-based safety monitoring technologies mainly use target detection algorithms such as YOLO and Faster R-CNN, which can effectively detect single-target attributes such as helmet wearing and protective equipment identification.

[0004] However, dangerous behaviors in construction scenarios are essentially the interaction of multiple agents under specific spatiotemporal conditions. Examples include workers entering the "death zone" of a crane's rotation radius and collision warnings for mobile equipment with temporary structures. Existing single-object detection frames are unable to capture these correlated risk characteristics, resulting in a hazard recognition rate of less than 45% (based on a benchmark test on the COCO-Safety dataset). Therefore, a method is urgently needed to address the low recognition rate of unsafe behaviors in construction environments, caused by the complex interactions of multiple objects and the inability of existing methods to quantify correlated risks. Summary of the Invention

[0005] In response to the above technical problems, the purpose of this application is to provide a method and system for analyzing unsafe behaviors in a construction work environment with multi-target correlation, aiming to solve the technical problem that the recognition rate of unsafe behaviors is low due to the complexity of multi-target interactions in the construction work environment and the inability of existing methods to quantify correlation risks.

[0006] The present invention provides a method for analyzing unsafe behaviors in a construction work environment with multi-objective associations, the method comprising:

[0007] Obtain the construction work environment image captured by the camera in real time;

[0008] Input the construction work environment image into a preset construction work environment unsafe behavior analysis model, identify behavioral safety risks through the construction work environment unsafe behavior analysis model and output the risk level; wherein the construction work environment unsafe behavior analysis model includes:

[0009] The input layer is used to detect various targets based on the construction work environment image using a target detection algorithm, segment the targets, and output the location information of the key points of each target; the key points of the target include the center point of the target and the upper and lower edge endpoints of the target;

[0010] Feature extraction layer, used to extract features of each target separately;

[0011] The association modeling layer is used to calculate the Euclidean distance of all key points between any two targets based on the position information of the key points and construct a distance matrix; and generate the feature fusion weight between any two targets based on the distance matrix;

[0012] The decision layer includes a fusion module and a classification layer. The fusion module is used to perform feature fusion using the feature fusion weights between any two targets and the features of each target. The classification layer identifies behavioral safety risks based on the fused features and outputs the risk level.

[0013] Furthermore, the types of targets include personnel, vehicles, suspended objects, equipment, work platforms and environmental hazardous areas.

[0014] Furthermore, the feature extraction layer includes a lightweight feature extraction module and a multi-scale feature fusion module, and the step of extracting features of each target separately includes:

[0015] For each target, extract its shallow features through the lightweight feature extraction module; wherein the lightweight feature extraction module includes a moving inverted residual convolution module and a coordinate attention mechanism module;

[0016] The shallow features are input into the multi-scale feature fusion module for feature fusion; wherein the multi-scale feature fusion module includes a void space pyramid pooling module and a feature pyramid module; wherein the output of the void space pyramid pooling module is the input of the feature pyramid module.

[0017] Furthermore, the step of calculating and outputting the risk level based on the fused features in the classification layer includes:

[0018] The fused features are input into the classification layer, which compresses the spatial dimension through global average pooling to retain the channel semantic information. The classification layer then passes through the dropout layer to randomly discard a specified proportion of neurons, and finally passes through the fully connected layer to output the risk level.

[0019] Furthermore, the step of generating feature fusion weights between any two targets based on the distance matrix includes:

[0020] Normalize the distance matrix and generate the feature fusion weight between any two targets according to the following formula:

[0021] ;

[0022] in, is the minimum Euclidean distance between target i and target j, and n is the summation index variable.

[0023] Furthermore, the fusion module is used to perform feature fusion using the feature fusion weights between any two targets and the features of each target, including:

[0024] Use the corresponding feature fusion weights to weight the features of the corresponding targets. The weighting method is: ;in, is the feature of target j;

[0025] Concatenate the weighted features.

[0026] Furthermore, in the target detection stage, the target detection algorithm adopts the YOLO model, which is trained separately. After the YOLO model is determined, the parameters are fixed and do not participate in subsequent network training. For the behavioral safety risk classification loss, the classification loss with ordinal relationship and the label smoothing loss to prevent model overfitting are adopted. The behavioral safety risk classification loss function ;

[0027] in, ;

[0028] ;

[0029] in, represents the behavioral safety risk classification loss, is the classification loss with ordinal relationship, is the label smoothing loss, BCE is the binary cross entropy loss function, y represents the risk level, Indicates that the sample belongs to level k or higher risk, is the Sigmod function, k represents the index of the current binary classification task, and its value range is ; w represents the weight vector, b represents the bias term, x is the input feature vector, and T represents the transpose; represents hot encoding, is a hyperparameter used to resolve the impact of annotation noise, Indicates the number of categories.

[0030] In a second aspect, an embodiment of the present application provides a multi-target associated unsafe behavior analysis system for a construction work environment, the system comprising:

[0031] An acquisition module is used to acquire the construction work environment image captured by the camera in real time;

[0032] An input module is configured to input a construction work environment image into a preset construction work environment unsafe behavior analysis model, identify behavioral safety risks through the construction work environment unsafe behavior analysis model, and output a risk level; wherein the construction work environment unsafe behavior analysis model includes:

[0033] The input layer is used to detect various targets based on the construction work environment image using a target detection algorithm, segment the targets, and output the location information of the key points of each target; the key points of the target include the center point of the target and the upper and lower edge endpoints of the target;

[0034] Feature extraction layer, used to extract features of each target separately;

[0035] The association modeling layer is used to calculate the Euclidean distance of all key points between any two targets based on the position information of the key points and construct a distance matrix; and generate the feature fusion weight between any two targets based on the distance matrix;

[0036] The decision layer includes a fusion module and a classification layer. The fusion module is used to perform feature fusion using the feature fusion weights between any two targets and the features of each target. The classification layer identifies behavioral safety risks based on the fused features and outputs the risk level.

[0037] Furthermore, the types of targets include personnel, vehicles, hanging objects, equipment, work platforms and environmental hazardous areas. The feature extraction layer includes a lightweight feature extraction module and a multi-scale feature fusion module. The step of extracting the features of each target separately includes:

[0038] For each target, extract its shallow features through the lightweight feature extraction module; wherein the lightweight feature extraction module includes a moving inverted residual convolution module and a coordinate attention mechanism module;

[0039] The shallow features are input into the multi-scale feature fusion module for feature fusion; wherein the multi-scale feature fusion module includes a void space pyramid pooling module and a feature pyramid module; wherein the output of the void space pyramid pooling module is the input of the feature pyramid module.

[0040] Furthermore, generating feature fusion weights between any two targets based on the distance matrix includes:

[0041] Normalize the distance matrix and generate the feature fusion weight between any two targets according to the following formula:

[0042] ;

[0043] in, is the minimum Euclidean distance between target i and target j, and n is the summation index variable.

[0044] The present invention provides a method for analyzing unsafe behaviors in a construction work environment with multiple targets associated. The method comprises: obtaining a construction work environment image captured in real time by a camera; inputting the construction work environment image into a preset construction work environment unsafe behavior analysis model, identifying behavioral safety risks through the construction work environment unsafe behavior analysis model and outputting risk levels; wherein the construction work environment unsafe behavior analysis model comprises: an input layer, used to detect various types of targets based on the construction work environment image through a target detection algorithm, segment the targets and output the position information of the key points of each target; the key points of the target include the center point of the target and the upper and lower edge endpoints of the target; a feature extraction layer, used to extract the features of each target separately; an association modeling layer, used to calculate the Euclidean distance of all key points between any two targets based on the position information of the key points, and construct a distance matrix; generating feature fusion weights between any two targets based on the distance matrix; a decision layer, the decision layer comprising a fusion module and a classification layer, the fusion module being used to perform feature fusion using the feature fusion weights between any two targets and the features of each target, and the classification layer identifying behavioral safety risks based on the fused features and outputting risk levels. Compared to traditional methods that only use the target center point and cannot reflect edge contact risks, such as overlap between equipment and the edge of a cavity, this invention extracts the target center point and the upper and lower edge endpoints of the target, and establishes the distance relationship between each point between the targets to calculate the feature fusion weight. Compared with traditional methods that only use the center point distance, the accuracy of dangerous behavior identification is improved, while effectively reducing the false alarm rate. This solves the technical problem of low unsafe behavior identification rate due to the complex interaction of multiple targets in construction work environments and the inability of existing methods to quantify correlation risks.

[0045] Compared with the traditional single-target detection combined with a rule engine, the "detection-association-decision-making" closed-loop architecture of the present invention reduces the overall false alarm rate in complex scenarios (such as nighttime welding operations where people, open flames, and flammable materials coexist). It achieves a technological leap from isolated target recognition to group behavior risk reasoning, and provides an explainable and highly responsive intelligent analysis solution for building safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the implementation. Obviously, the drawings described below are only some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0047] Figure 1 This is a flowchart of a method for analyzing unsafe behaviors in a construction work environment with multi-objective associations provided by an embodiment of the present application;

[0048] Figure 2 This is a schematic diagram of the structure of an unsafe behavior analysis model for a construction work environment provided by an embodiment of the present application;

[0049] Figure 3 This is a simplified structural diagram of a multi-target associated construction work environment unsafe behavior analysis system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0051] Those skilled in the art will understand that, unless expressly stated otherwise, the singular forms "a", "an", "above", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of this application refers to the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any module and all combinations of one or more associated listed items.

[0052] Those skilled in the art will understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and, unless specifically defined as such, will not be interpreted in an idealized or overly formal sense.

[0053] like Figure 1 、 Figure 2 As shown, the embodiment of the present application provides a method for analyzing unsafe behaviors in a construction work environment with multi-objective associations, the method comprising:

[0054] S1. Obtaining a real-time image of the construction environment captured by a camera;

[0055] S2. Inputting the construction work environment image into a preset construction work environment unsafe behavior analysis model, identifying behavioral safety risks through the construction work environment unsafe behavior analysis model and outputting the risk level; wherein the construction work environment unsafe behavior analysis model includes:

[0056] The input layer 11 is used to detect various targets based on the construction work environment image using a target detection algorithm, segment the targets, and output the location information of the key points of each target; the key points of the target include the center point of the target and the upper and lower edge endpoints of the target;

[0057] Feature extraction layer 12, used to extract features of each target separately;

[0058] The association modeling layer 13 is used to calculate the Euclidean distance of all key points between any two targets based on the position information of the key points and construct a distance matrix; and generate feature fusion weights between any two targets based on the distance matrix;

[0059] The decision layer 14 includes a fusion module and a classification layer. The fusion module is used to perform feature fusion using the feature fusion weights between any two targets and the features of each target. The classification layer identifies behavioral safety risks based on the fused features and outputs the risk level.

[0060] In step S1, cameras, such as 4K wide-angle cameras with H.265 encoding and low-light enhancement, are deployed in key areas of the construction site, such as high-altitude work areas and hoisting areas. These cameras capture real-time images of the construction environment for subsequent analysis and processing.

[0061] In the above step S2, a conventional target detection model, such as YOLOv11, is used in combination with the MobileNetV3 lightweight backbone network and customized Anchors (such as long hanging object Anchors and small-sized hole Anchors) to detect 6 types of targets, including personnel, vehicles, hanging objects, equipment, work platforms, and environmental hazardous areas, and output the target frame. After obtaining the target frame, the center point of the target frame (the target frame here refers to the rectangular target frame) can be extracted as the center point of the target, and the upper and lower edge endpoints of the target frame can be extracted as the upper and lower edge endpoints of the target. The upper and lower edge endpoints of the target frame are the four corner points of the target frame. By extracting the center point of the target, the upper and lower edge endpoints of the target (the center point plus the lower edge endpoint, a total of 5 geometric key points) lay the foundation for subsequent analysis. The association modeling layer is to realize target space association modeling. Specifically, based on the position information of the 5 key points, the Euclidean distance of all key points between any two targets is calculated, and the distance matrix D is constructed. Where R represents a set of real numbers; that is, the elements in the distance matrix D, or distance information, are all real numbers. N represents the number of objects detected in the image. D is a three-dimensional matrix with dimensions N×N×5, representing the Euclidean distance of five key points between any two objects. Distance matrix D is then normalized, and the Softmax function is used to generate feature fusion weights between any two objects. The decision layer performs feature fusion based on the feature fusion weights and the object features. Finally, based on the fused features, it identifies the behavioral safety risk and outputs a risk level, which ranges from low risk, medium risk, to high risk.

[0062] Because construction accidents are often caused by multi-target interactions, such as when a suspended object is too close to a worker, traditional methods, which only use the center point of the target, cannot reflect edge contact risks, such as when equipment overlaps with the edge of a cavity. This invention extracts the center point of the target and the upper and lower edge endpoints of the target, and establishes distance relationships between each point between the targets to calculate feature fusion weights. Compared to existing methods that only use center point distances, this method improves the accuracy of dangerous behavior identification and effectively reduces false alarm rates. This solves the technical problem of low unsafe behavior recognition rates in construction work environments due to the complex multi-target interactions and the inability of existing methods to quantify correlation risks.

[0063] Compared with the traditional single-target detection combined with a rule engine, the "detection-association-decision-making" closed-loop architecture of the present invention reduces the overall false alarm rate in complex scenarios (such as nighttime welding operations where people, open flames, and flammable materials coexist). It achieves a technological leap from isolated target recognition to group behavior risk reasoning, and provides an explainable and highly responsive intelligent analysis solution for building safety management.

[0064] In one embodiment, the types of targets include personnel, vehicles, hanging objects, equipment, work platforms, and environmental hazardous areas.

[0065] In one embodiment, the feature extraction layer includes a lightweight feature extraction module and a multi-scale feature fusion module, and the step of extracting features of each target separately includes:

[0066] For each target, extract shallow features thereof through the lightweight feature extraction module; wherein the lightweight feature extraction module includes a mobile inverted bottleneck convolution module (full name: Mobile Inverted Bottleneck Convolution; English abbreviation: MBConv) and a coordinate attention mechanism module (full name: CoordinateAttention; English abbreviation: CA);

[0067] The shallow features are input into the multi-scale feature fusion module for feature fusion; wherein the multi-scale feature fusion module includes an atrous spatial pyramid pooling module (English full name: Atrous Spatial Pyramid Pooling; English abbreviation: ASPP) and a feature pyramid module (English full name: Feature Pyramid Networks; English abbreviation: FPN); wherein the output of the atrous spatial pyramid pooling module is the input of the feature pyramid module.

[0068] In an embodiment of the present application, the feature extraction layer mainly includes a lightweight feature extraction module and a multi-scale feature fusion module. The lightweight feature extraction module provides an efficient feature basis for the subsequent ASPP multi-scale fusion, and the multi-scale features output by the void space pyramid pooling module serve as the input of the feature pyramid module. The two work together to improve spatial perception capabilities. The lightweight feature extraction module mainly includes a moving inverted residual convolution module and a coordinate attention mechanism module. The moving inverted residual convolution module is a depth-separable convolution block in MobileNet V2, containing an inverted residual structure, and the coordinate attention mechanism module encodes spatial position information in the horizontal and vertical directions through coordinate decomposition. The moving inverted residual convolution module reduces the number of parameters on the basis of the backbone network to avoid computational redundancy. Subsequently, the coordinate attention mechanism module is used to enhance the perception of the target spatial distribution (such as the relative position of the hanging object and the person), thereby improving the positioning sensitivity of key targets (such as workers near the hole). The lightweight feature extraction module, consisting primarily of the first three layers, utilizes moving inverted residual convolution coupled with a coordinate attention mechanism to balance model accuracy and computational efficiency. This module's design effectively reduces computational effort by 30% (compared to standard convolution) and improves object location sensitivity by 15%. This module not only meets real-time surveillance requirements (>25 FPS (Frames Per Second)), but also avoids missing critical targets (such as small holes). More specifically, the lightweight feature extraction module performs a preliminary encoding of each target's ROI (Region of Interest) using a 3×3 convolution. This encoding is then fed into a six-layer convolutional network, where the first three layers utilize moving inverted residual convolution coupled with a coordinate attention mechanism.

[0069] The multi-scale feature fusion module primarily consists of a dilated spatial pyramid pooling module and a feature pyramid module. The dilated spatial pyramid pooling module uses convolutional layers with varying dilation rates to concurrently extract multi-scale features (e.g., the large size of a hoisted object versus the details of a welding torch). The feature pyramid constructs three convolutional layers of varying depth: a high-resolution detail layer, a mid-level semantic layer, and a low-level global layer. These layers correspond to local details, semantic information, and global context, respectively. This extracts both local details (e.g., welding torch sparks) and global semantic information, enhancing the model's robustness to scale variations (e.g., large hoisted objects versus small holes) and occlusions (e.g., hoisted objects obscuring workers). Due to the large scale disparity of objects in building scenes (e.g., work platforms versus rebar), multi-scale feature coverage is required. Therefore, the dilated spatial pyramid pooling module addresses object occlusions (e.g., hoisted objects obscuring workers), while the feature pyramid module enhances small object detection. This module is designed to adapt to the dynamic changes in complex building scenes, such as the overlapping risk of hoisted object movement and human relocation. This improves the accuracy of multi-object individual feature detection and reduces false positives in occluded scenes. More specifically, dilated spatial pyramid pooling uses parallel convolutions with dilation rates of {1, 6, 12, 18} to capture features with different receptive fields. Dilated spatial pyramid pooling combines dilated convolutions with feature pyramids to expand the receptive field without increasing the number of parameters. Feature pyramids fuse multi-granular features across multiple levels.

[0070] In one embodiment, the step of calculating and outputting the risk level based on the fused features by the classification layer includes:

[0071] The fused features are input into the classification layer, which compresses the spatial dimension through global average pooling to retain the channel semantic information. Then, a dropout layer is used to randomly drop a specified proportion of neurons to prevent the model from overfitting. Finally, a fully connected layer is used to output the risk level.

[0072] In the embodiment of the present application, specifically, the classification layer first compresses the spatial dimension to 1*1 through global average pooling GAP (Global AveragePooling), retaining the channel semantic information, and then randomly discards 30% of the neurons through the dropout layer to prevent the model from overfitting. Finally, through the fully connected layer, a 3-dimensional vector is output, namely the three types of risk: low, medium and high, and the probability is normalized using Softmax. Specifically, a probability greater than 0.7 is judged as high risk, and an early warning message is triggered (such as triggering an audible and visual alarm) and pushed to the management personnel. For probability values ​​of 0.4-0.7, the system records the log and prompts the on-site supervisor to review. When the probability value is less than 0.4, the system judges it as low risk and the system does not perform any processing. Through the above process, a hierarchical response is achieved, and the early warning sensitivity and false alarm rate are also balanced.

[0073] In one embodiment, the step of generating feature fusion weights between any two targets based on the distance matrix includes:

[0074] Normalize the distance matrix and generate the feature fusion weight between any two targets according to the following formula:

[0075] ;

[0076] in, is the minimum Euclidean distance between target i and target j, and n is the summation index variable.

[0077] This formula is the Softmax function, that is, the feature fusion weights between any two targets are generated by the Softmax function.

[0078] In one embodiment, the step of the fusion module for performing feature fusion using the feature fusion weight between any two targets and the features of each target includes:

[0079] Use the corresponding feature fusion weights to weight the features of the corresponding targets. The weighting method is: ;in, is the feature of target j;

[0080] Concatenate the weighted features.

[0081] In the present invention, the fusion module can specifically adopt a concatenated convolution fusion module (full name: ConcatConvolution Fusion; English abbreviation: CCF). The concatenated convolution fusion module completes the fusion of different target features, that is, The features of each detected target are concatenated along the channel dimension. Specifically, for target i, its features are concatenated with the features of the other N-1 targets, and the concatenation dimension is the channel dimension. Assuming that the feature dimension of each target is C×H×W (C is the number of channels, H is the height, and W is the width), the concatenated feature dimension is (N×C)×H×W. When concatenating, the features of the corresponding targets are weighted using the corresponding feature fusion weights. The weighting method is: ;in, The weighted features are then concatenated. For example, the first target, A, has a correlation of 1 with itself, so its feature information, FA, is multiplied by 1, resulting in [FA]. The second target, B, has a weighting coefficient of 0.6 with A. Therefore, B's feature information, FB, is multiplied by the weighting coefficient 0.6, resulting in the concatenated information, [FA, FB*0.6]. Similarly, A and the third target also have a weighting coefficient, and the concatenation is performed in this manner. A 1×1 convolutional layer is then used to compress the concatenated features, converting them back to a C×H×W dimension while preserving the critical interaction information. This 1×1 convolutional layer can be considered a feature transformation layer, which fuses and compresses the concatenated features through learning, enabling the output features to better represent the spatial interaction risk coefficients between different targets and strengthen the representation of high-risk associated targets. Compared with the traditional method that only uses the center point, judging the distance between the upper and lower edges and the center point can more accurately reflect the contact risk. In the process of dynamic weight allocation, the targets with close distances contribute more to feature fusion, which helps to improve the accuracy of dangerous behavior identification.

[0082] In one embodiment, during the target detection phase, the target detection algorithm uses the YOLO model (full name: You Only Look Once). The YOLO model is trained separately. After the YOLO model is determined, the parameters are fixed and do not participate in subsequent network training. For the behavioral safety risk classification loss, a classification loss with an ordinal relationship and a label smoothing loss are used to prevent the model from overfitting. The total loss function of the behavioral safety risk classification is :

[0083] in, ;

[0084] ;

[0085] in, represents the behavioral safety risk classification loss, is the classification loss with ordinal relationship, is the label smoothing loss, BCE is the binary cross entropy loss function, y represents the risk level, Indicates that the sample belongs to level k or higher risk, is the Sigmod function, k represents the index of the current binary classification task, and its value range is ; w represents the weight vector, b represents the bias term, x is the input feature vector, and T represents the transpose; represents hot encoding, is a hyperparameter used to resolve the impact of annotation noise, Indicates the number of categories.

[0086] In the embodiment of the present application, specifically, in terms of model loss function design, the target detection in the first stage adopts the existing deep learning target detection algorithm YOLO, such as YOLOv11. This stage mainly completes the detection and segmentation of various targets. The loss function of this part includes bounding box regression loss and classification loss, and this part is trained separately. After the YOLO model is determined, the parameters are fixed and do not participate in subsequent network training. As for the classification loss of behavioral safety risks, the present invention proposes a classification loss with an ordinal relationship and a label smoothing loss to prevent model overfitting, wherein the classification loss with an ordinal relationship for: This ordinal classification loss function decomposes the ordered K-class classification task into K−1 binary classification tasks, enhancing classification robustness and reducing the overall false positive rate. For example, for the three levels of low (0), medium (1), and high (2), two binary classification problems are defined:

[0087] Task 1: Determine whether the sample is ≥ medium risk (i.e., label 1 or 2).

[0088] Task 2: Determine whether the sample is high risk (i.e., label 2).

[0089] Each task uses an independent BCE loss, so the final total loss is the sum of the losses of each task, that is, .

[0090] Label smoothing loss for: ;

[0091] Among them, the hyperparameters Mainly to eliminate the impact of annotation noise, set ;

[0092] Therefore, the total loss function of behavioral safety risk classification is: .

[0093] In summary, the modules of this application form an organic whole through a collaborative mechanism. A lightweight backbone network ensures real-time processing capabilities, while dilated spatial pyramid pooling and feature pyramid compensate for multi-scale detection accuracy. Geometric association modeling empowers spatial risk quantification, and end-to-end training deeply couples feature extraction with classification decisions. For example, when a suspended object moves, the moving inverted residual convolution and coordinate attention mechanism modules rapidly locate the target, while dilated spatial pyramid pooling captures multi-scale contours under motion blur. The feature pyramid enhances the recognition of small warning signs, and a geometric weighting mechanism accurately determines their spatial relationship with surrounding personnel. Finally, the classifier triggers a graded warning based on the weighted concatenated features. This closed-loop "detection-association-decision" architecture reduces the overall false alarm rate by 20% in complex scenarios (such as nighttime welding operations where personnel, open flames, and flammable materials coexist) compared to traditional single-target detection combined with a rule engine. This represents a technological leap from isolated target recognition to group risk reasoning, providing an interpretable and highly responsive intelligent analysis solution for building safety management.

[0094] This invention addresses the challenges of real-time, precision, and interpretability in risk analysis of multi-target interactions in architectural scenarios through the collaborative design of lightweight feature extraction, multi-scale fusion, target association modeling, and end-to-end classification. This approach reduces computational overhead while improving the accuracy and response speed of dangerous behavior identification. The modules are integrated through parameter sharing, feature transfer, and feedback mechanisms. Compared to traditional single-target detection and rule-based engine approaches, the overall performance (FPS, accuracy, and false alarm rate) in complex scenarios is significantly improved.

[0095] Field tests have shown that this method can accurately identify correlated hazardous behaviors in complex construction scenarios, reaching 91.2% accuracy—a 2.1-fold improvement over traditional methods. This technology provides a new paradigm for safety situational awareness in intelligent construction, potentially reducing preventable accidents by 28% and driving the construction industry's transformation and upgrade towards intrinsic safety.

[0096] In one embodiment, the input layer is further configured to output a target attribute label; wherein the target attribute label includes target type and semantic information;

[0097] The feature fusion weight is adjusted based on the target attribute label.

[0098] In the embodiment of the present application, the personnel attribute labels include whether they wear a safety helmet, whether they wear reflective clothing, and whether they wear gloves. Vehicles include cranes, dump trucks, private cars and other special vehicles, and hoisted objects include counterweights, steel bars, wooden boards, etc. Work platforms include static load test platforms, drilling platforms, etc. Equipment devices include welding guns, oxygen tanks, acetylene tanks, etc. Environmental hazardous areas include steep slopes, exposed steel bars, holes, etc. The feature fusion weight is adjusted based on the target attribute label. If the target attribute label indicates that it is more likely to cause safety accidents, the greater the adjustment range of the feature fusion weight, the more accurate the identification of dangerous behaviors will be.

[0099] like Figure 3 As shown, the embodiment of the present application also provides a multi-target associated unsafe behavior analysis system for a construction work environment, the system comprising:

[0100] Acquisition module 1, used to acquire the construction work environment image captured by the camera in real time;

[0101] Input module 2 is used to input the construction work environment image into a preset construction work environment unsafe behavior analysis model, identify behavioral safety risks through the construction work environment unsafe behavior analysis model and output the risk level; wherein the construction work environment unsafe behavior analysis model includes:

[0102] The input layer is used to detect various targets based on the construction work environment image using a target detection algorithm, segment the targets, and output the location information of the key points of each target; the key points of the target include the center point of the target and the upper and lower edge endpoints of the target;

[0103] Feature extraction layer, used to extract features of each target separately;

[0104] The association modeling layer is used to calculate the Euclidean distance of all key points between any two targets based on the position information of the key points and construct a distance matrix; and generate the feature fusion weight between any two targets based on the distance matrix;

[0105] The decision layer includes a fusion module and a classification layer. The fusion module is used to perform feature fusion using the feature fusion weights between any two targets and the features of each target. The classification layer identifies behavioral safety risks based on the fused features and outputs the risk level.

[0106] In one embodiment, the types of targets include personnel, vehicles, hanging objects, equipment, work platforms, and environmental hazardous areas. The feature extraction layer includes a lightweight feature extraction module and a multi-scale feature fusion module. The feature extraction of each target includes:

[0107] For each target, extract its shallow features through the lightweight feature extraction module; wherein the lightweight feature extraction module includes a moving inverted residual convolution module and a coordinate attention mechanism module;

[0108] The shallow features are input into the multi-scale feature fusion module for feature fusion; wherein the multi-scale feature fusion module includes a void space pyramid pooling module and a feature pyramid module; wherein the output of the void space pyramid pooling module is the input of the feature pyramid module.

[0109] In one embodiment, generating a feature fusion weight between any two targets based on a distance matrix includes:

[0110] Normalize the distance matrix and generate the feature fusion weight between any two targets according to the following formula:

[0111] ;

[0112] in, is the minimum Euclidean distance between target i and target j, and n is the summation index variable.

[0113] In one embodiment, the classification layer calculates and outputs the risk level based on the fused features, including:

[0114] The fused features are input into the classification layer, which compresses the spatial dimension through global average pooling to retain the channel semantic information. The classification layer then passes through the dropout layer to randomly discard a specified proportion of neurons, and finally passes through the fully connected layer to output the risk level.

[0115] In one embodiment, the fusion module is configured to perform feature fusion using the feature fusion weight between any two targets and the features of each target, including:

[0116] Use the corresponding feature fusion weights to weight the features of the corresponding targets. The weighting method is: ;in, is the feature of target j;

[0117] Concatenate the weighted features.

[0118] In one embodiment, in the target detection stage, the target detection algorithm adopts the YOLO model, and the YOLO model is trained separately. After the YOLO model is determined, the parameters are fixed and do not participate in subsequent network training. For the behavioral safety risk classification loss, the classification loss with ordinal relationship and the label smoothing loss to prevent model overfitting are adopted. The behavioral safety risk classification loss function ;

[0119] in, ;

[0120] ;

[0121] in, represents the behavioral safety risk classification loss, is the classification loss with ordinal relationship, is the label smoothing loss, BCE is the binary cross entropy loss function, y represents the risk level, Indicates that the sample belongs to level k or higher risk, is the Sigmod function, k represents the index of the current binary classification task, and its value range is ; w represents the weight vector, b represents the bias term, x is the input feature vector, and T represents the transpose; represents hot encoding, is a hyperparameter used to resolve the impact of annotation noise, Indicates the number of categories.

[0122] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and RAM bus dynamic RAM (RDRAM).

[0123] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0124] The above description is only a preferred embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A multi-objective association unsafe behavior analysis method for construction work environment, characterized by: The method comprises: Obtain the construction work environment image captured by the camera in real time; Input the construction work environment image into a preset construction work environment unsafe behavior analysis model, identify behavioral safety risks through the construction work environment unsafe behavior analysis model and output the risk level; wherein the construction work environment unsafe behavior analysis model includes: The input layer is used to detect various targets based on the construction work environment image using a target detection algorithm, segment the targets, and output the location information of the key points of each target; the key points of the target include the center point of the target and the upper and lower edge endpoints of the target; Feature extraction layer, used to extract features of each target separately; The association modeling layer is used to calculate the Euclidean distance of all key points between any two targets based on the position information of the key points and construct a distance matrix; and generate the feature fusion weight between any two targets based on the distance matrix; The decision layer includes a fusion module and a classification layer. The fusion module is used to perform feature fusion using the feature fusion weight between any two targets and the features of each target. The classification layer identifies behavioral safety risks based on the fused features and outputs the risk level. The step of generating feature fusion weights between any two targets based on the distance matrix includes: Normalize the distance matrix and generate the feature fusion weight between any two targets according to the following formula: ； in, is the minimum Euclidean distance between target i and target j, and n is the summation index variable; The steps of the fusion module for performing feature fusion using the feature fusion weights between any two targets and the features of each target include: Use the corresponding feature fusion weights to weight the features of the corresponding targets. The weighting method is: ;in, is the feature of target j; The weighted features are spliced; when i=1, for target 1, the splicing method is , ];in, is the characteristic of target 1, is the feature fusion weight between target 1 and target 2, is the characteristic of target 2, is the feature fusion weight between target 1 and target N, is the characteristic of target N; N is the sum of the number of targets 1 and the targets around target 1; The concatenated features are fused and compressed through a 1*1 convolutional layer.

2. The multi-objective association construction work environment unsafe behavior analysis method according to claim 1 is characterized in that: Types of targets include personnel, vehicles, suspended objects, equipment, work platforms and environmental hazardous areas.

3. The multi-objective association construction work environment unsafe behavior analysis method according to claim 2 is characterized in that: The feature extraction layer includes a lightweight feature extraction module and a multi-scale feature fusion module. The step of extracting the features of each target separately includes: For each target, extract its shallow features through the lightweight feature extraction module; wherein the lightweight feature extraction module includes a moving inverted residual convolution module and a coordinate attention mechanism module; The shallow features are input into the multi-scale feature fusion module for feature fusion; wherein the multi-scale feature fusion module includes a void space pyramid pooling module and a feature pyramid module; wherein the output of the void space pyramid pooling module is the input of the feature pyramid module.

4. The multi-objective association construction work environment unsafe behavior analysis method according to claim 1 is characterized in that: The steps of calculating and outputting the risk level based on the fused features in the classification layer include: The fused features are input into the classification layer, which compresses the spatial dimension through global average pooling to retain the channel semantic information. The classification layer then passes through the dropout layer to randomly discard a specified proportion of neurons, and finally passes through the fully connected layer to output the risk level.

5. The multi-objective association construction work environment unsafe behavior analysis method according to claim 1 is characterized in that: In the target detection stage, the target detection algorithm adopts the YOLO model, which is trained separately. After the YOLO model is determined, the parameters are fixed and do not participate in subsequent network training. For the behavioral safety risk classification loss, the classification loss with ordinal relationship and the label smoothing loss to prevent model overfitting are adopted. The behavioral safety risk classification loss function ; in, ; ； in, represents the behavioral safety risk classification loss, is the classification loss with ordinal relationship, is the label smoothing loss, BCE is the binary cross entropy loss function, y represents the risk level, Indicates that the sample belongs to level k or higher risk, is the Sigmod function, k represents the index of the current binary classification task, and its value range is ; w represents the weight vector, b represents the bias term, x is the input feature vector, and T represents the transpose; represents hot encoding, is a hyperparameter used to resolve the impact of annotation noise, Indicates the number of categories.

6. A multi-objective associated construction work environment unsafe behavior analysis system, characterized by: The system is used to execute the multi-target associated construction work environment unsafe behavior analysis method according to any one of claims 1 to 5, and the system comprises: An acquisition module is used to acquire the construction work environment image captured by the camera in real time; An input module is configured to input a construction work environment image into a preset construction work environment unsafe behavior analysis model, identify behavioral safety risks through the construction work environment unsafe behavior analysis model, and output a risk level; wherein the construction work environment unsafe behavior analysis model includes: The input layer is used to detect various targets based on the construction work environment image using a target detection algorithm, segment the targets, and output the location information of the key points of each target; the key points of the target include the center point of the target and the upper and lower edge endpoints of the target; Feature extraction layer, used to extract features of each target separately; The association modeling layer is used to calculate the Euclidean distance of all key points between any two targets based on the position information of the key points and construct a distance matrix; and generate the feature fusion weight between any two targets based on the distance matrix; The decision layer includes a fusion module and a classification layer. The fusion module is used to perform feature fusion using the feature fusion weights between any two targets and the features of each target. The classification layer identifies behavioral safety risks based on the fused features and outputs the risk level.

7. The multi-objective association construction work environment unsafe behavior analysis system according to claim 6 is characterized in that: Target types include personnel, vehicles, hanging objects, equipment, work platforms, and hazardous areas. The feature extraction layer includes a lightweight feature extraction module and a multi-scale feature fusion module. The step of extracting features for each target separately includes: For each target, extract its shallow features through the lightweight feature extraction module; wherein the lightweight feature extraction module includes a moving inverted residual convolution module and a coordinate attention mechanism module; The shallow features are input into the multi-scale feature fusion module for feature fusion; wherein the multi-scale feature fusion module includes a void space pyramid pooling module and a feature pyramid module; wherein the output of the void space pyramid pooling module is the input of the feature pyramid module.

8. The multi-objective association construction work environment unsafe behavior analysis system according to claim 6 is characterized in that: The generating of feature fusion weights between any two targets based on the distance matrix includes: Normalize the distance matrix and generate the feature fusion weight between any two targets according to the following formula: ； in, is the minimum Euclidean distance between target i and target j, and n is the summation index variable.

Images