Accident potential troubleshooting method and system based on scene recognition

Abstract

The invention relates to the technical field of building construction safety management and artificial intelligence image recognition, in particular to an accident potential troubleshooting method and system based on scene recognition, and the method comprises the steps: collecting and preprocessing multi-view video stream data in real time, and carrying out the semantic segmentation of the preprocessed multi-view video stream, extracting spatial semantic information including construction areas, operation behaviors, key equipment and other elements; on the basis, by combining a human body posture and behavior recognition model, the potential safety hazards such as no safety belt wearing, non-standard object stacking and personnel border crossing operation in aerial work are automatically detected, a dynamic risk assessment model is constructed based on historical experience and field states, and grading is performed based on a risk assessment model calculation structure. According to the hidden danger level, the system can automatically trigger corresponding multi-level early warning response mechanisms such as voice, images or platform pushing, and closed-loop management of hidden danger discovery, early warning and intervention is realized. The problems that in the prior art, hidden danger recognition depends on manual work, accuracy is poor, and response lags behind are solved.

Description

Technical Field

[0001] This invention relates to the fields of construction safety management and artificial intelligence image recognition technology, specifically to a method and system for identifying potential safety hazards based on scene recognition. Background Technology

[0002] With the acceleration of urbanization and the expansion of infrastructure construction, the complexity and risks of construction sites are becoming increasingly prominent, and construction safety has become a focus of attention for all sectors of society. Statistics show that construction accidents caused by management oversights, operational violations, and environmental factors still occur frequently, seriously threatening the lives of construction workers and causing substantial property losses.

[0003] Currently, safety supervision at most construction sites still relies on manual inspections and static rule reviews. However, these methods have significant limitations, such as delayed information feedback, limited regulatory coverage, and strong subjectivity in hazard identification. Although some construction companies have deployed video surveillance systems, these existing systems only have image acquisition and storage functions and lack the ability to deeply understand complex dynamic scenes, as well as the ability to automatically identify and judge violations and potential hazards.

[0004] In recent years, with the rapid development of deep learning and computer vision technologies, key algorithms such as image recognition, behavior detection, and semantic segmentation have made breakthroughs in fields such as industry and transportation, promoting the penetration and integration of artificial intelligence (AI) technology into the field of construction safety management. However, research combining scene recognition with hazard analysis is still in its early stages and faces the following main problems:

[0005] 1. Complex construction scene structure: Construction sites involve numerous tasks, dense personnel, and severe visual obstruction, making conventional identification methods prone to misjudgment or missed judgment in such environments;

[0006] 2. Weak spatiotemporal correlation and poor dynamic understanding: Most systems lack analysis of the temporal continuity of construction activities, making it difficult to identify safety hazards with hidden and trend-like characteristics;

[0007] 3. Lagging early warning mechanism and insufficient response capability: The existing solution lacks a multi-level response mechanism that is automatically triggered based on the level of hazard, making it difficult to achieve real-time intervention and dynamic alerts for safety risks.

[0008] Therefore, there is an urgent need to propose a new method and system for identifying potential hazards at construction sites that combines semantic perception, dynamic understanding, and risk assessment capabilities, in order to improve the intelligence, real-time and automation of on-site safety management and control, and to ensure the safety of construction personnel and the smooth progress of the construction process. Summary of the Invention

[0009] The purpose of this invention is to address the problems and improvement needs of the existing technology by proposing a method and system for identifying potential safety hazards based on scene recognition, so as to solve at least one of the problems existing in the existing technology.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] A method for identifying potential safety hazards based on scene recognition includes the following steps:

[0012] S1. Image Acquisition and Processing: Real-time acquisition of multi-view video streams of key areas at the construction site; image preprocessing and keyframe extraction of the video streams to generate a structured, standardized image sequence; the image sequence includes a timestamp and acquisition location information.

[0013] S2. Parallel extraction of multi-dimensional risk features, specifically:

[0014] S21. Scene semantic segmentation and risk weight mapping: Based on the image sequence obtained in step S1, pixel-level parsing is performed through a deep learning semantic segmentation model to output a semantic segmentation map containing semantic categories of construction scenes; and a predefined region importance weight value is mapped to each semantic category in the semantic segmentation map;

[0015] S22. Personnel Behavior Recognition and Feature Extraction: Target detection technology is used to locate construction workers in the image sequence, and key points of the human skeleton are extracted to construct skeletal features; based on the skeletal features and semantic segmentation map, the behavioral risk probability of dangerous actions is calculated through an action discriminant function. The skeletal features and behavioral risk probabilities constitute personnel behavior features; the key points of the human skeleton include major parts of the human body such as the head, torso, and limb joints, used to describe the spatial posture of the personnel at the current moment.

[0016] S23. Environmental risk factor extraction: Based on image sequences and / or external sensor data, identify the current construction environment status and output environmental risk factors;

[0017] S3. Multimodal Feature Fusion and Preliminary Hazard Identification: The semantic segmentation map from step S21, the personnel behavior features from step S22, and the collected location information from step S1 are aligned and fused. The correlation between different features is modeled using a self-attention mechanism to obtain fused features. Based on the fused features, the hazard existence confidence is calculated using a hazard discrimination function. When the hazard existence confidence exceeds a first preset threshold, a hazard is determined to exist and the process proceeds to the next step.

[0018] S4. Comprehensive assessment of hazard level: For the hazards identified in step S3, a risk assessment model is constructed based on behavioral risk probability, regional importance weight value and environmental risk factors. The model uses a multi-factor weighted algorithm to calculate the weight of each risk factor and outputs the total risk score S of the hazard. Then, the total risk score is classified according to the second preset threshold to determine the hazard level.

[0019] S5. Tiered Response and Early Warning: Based on the hazard level determined in step S4, generate a hazard heat map and trigger the corresponding multi-level early warning mechanism.

[0020] Furthermore, the key area in step S1 refers to the area on the construction site related to the density of personnel activity, the frequency of equipment operation, and the distribution of potential hazards; the key area is determined based on the construction organization design, safety plan, or site survey results, and typically includes, but is not limited to:

[0021] High-altitude work areas: such as scaffolding, work platforms, tower crane operating areas, and other areas where there is a risk of falling;

[0022] Edge and opening areas: such as the edges of floors that are not completely enclosed, the perimeter of foundation pits, and passageway openings;

[0023] Large equipment operating areas: such as the area within the slewing radius of tower cranes, hoisting operation areas, and mechanical equipment passage routes, etc.

[0024] Material storage and processing areas: such as steel bar processing sheds, carpentry processing areas, and temporary material storage areas;

[0025] Hot work and temporary construction areas: such as welding work sites, cutting work sites, and areas near temporary electrical equipment.

[0026] Furthermore, the deep learning semantic segmentation model in step S21 is a multi-scale semantic segmentation network based on the Transformer structure; the dataset used to train the multi-scale semantic segmentation network contains pixel-level annotations of at least 20 types of key construction targets, and data augmentation strategies such as random occlusion, color perturbation, and weather simulation are adopted in the training.

[0027] Furthermore, in step S22, the action discrimination function is:

[0028] ;

[0029] Wherein, P is the set of key points in the skeleton feature. This indicates the change in the angle between key points. The rate of change of the skeleton segment length. This is the semantic segmentation map of the current frame. This is the classification model obtained through training.

[0030] Furthermore, step S22 also includes constructing a pose sequence of consecutive frames based on the key point set, and identifying implicit risk behaviors through a dynamic template matching mechanism; the dynamic template matching mechanism is achieved by calculating the cosine similarity between the pose sequence and a pre-stored dangerous action template, and the formula for calculating the cosine similarity is: .

[0031] Furthermore, the specific method for feature alignment and fusion in step S3 is as follows:

[0032] The semantic segmentation map, the personnel behavior features, and the collected location information are mapped to a high-dimensional space through an embedding function and then concatenated. The dependencies between features are modeled by the self-attention mechanism of the Transformer encoder, and then weighted by the channel attention mechanism to obtain the fused features. .

[0033] Furthermore, the risk assessment model in step S4 is as follows:

[0034] ;

[0035] in, This represents the probability of behavioral risk obtained in step S22; The region importance weight values ​​mapped in step S21; , , All are adjustable weight parameters, and can be selected in [0,1] or according to the actual situation.

[0036] Furthermore, in step S4, the total risk score S is classified into levels according to the second preset threshold to determine the hazard level K, as follows:

[0037] .

[0038] Furthermore, the multi-level early warning mechanism in step S5 includes:

[0039] against Level 3 hazards trigger audible and visual alarms and real-time push notifications to multiple terminals;

[0040] against Level 3 hidden dangers trigger visual alarms and work orders on the management terminal;

[0041] against Level 1 hazard generation background inspection record.

[0042] An accident hazard investigation system based on scene recognition includes a multi-view data acquisition module, a multi-dimensional risk feature parallel extraction module, a multi-modal feature fusion and hazard preliminary judgment module, a hazard level comprehensive assessment module, and a graded response and early warning module. These modules are used to implement the aforementioned accident hazard investigation method based on scene recognition.

[0043] By adopting the above technical solution, the present invention has the following beneficial effects:

[0044] 1. By combining deep semantic segmentation and behavior detection technologies, this invention can simultaneously identify the work area, determine the type of work, and detect personnel behavior in complex construction scenarios, significantly improving the comprehensiveness and accuracy of safety hazard identification.

[0045] 2. This invention takes multimodal features, including semantic segmentation maps, personnel behavior features, environmental risk factors, and collection location information, as input, and combines self-attention and channel attention mechanisms to fuse them to generate fused features. Based on these fused features, a custom hazard discrimination function is used to complete the initial hazard judgment, thereby improving the ability to identify dangerous situations in multi-factor interaction scenarios.

[0046] 3. This invention calculates a total risk score by designing a risk assessment model that combines historical risk data, operational status, and environmental context. Based on the total risk score, the severity of potential hazards is classified and managed in a hierarchical manner, thereby achieving differentiated response and graded intervention. Attached Figure Description

[0047] Figure 1 This is a flowchart of an accident hazard investigation method based on scene recognition according to the present invention;

[0048] Figure 2 Semantic segmentation flowchart of the example;

[0049] Figure 3 Example: Flowchart of personnel behavior recognition and feature extraction;

[0050] Figure 4 This is an example of integrating multimodal features into the hazard level classification and response early warning process. Detailed Implementation

[0051] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments. The described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] like Figure 1 The accident hazard investigation method based on scene recognition provided in this embodiment includes the following steps:

[0053] Step S1, image acquisition and processing, is implemented as follows:

[0054] S11. Image Acquisition: To achieve dynamic hazard identification at the construction site, this embodiment constructs a multi-view video image acquisition system to acquire multi-view video streams. The multi-view video image acquisition system includes high-definition cameras deployed in key areas such as tower crane areas, edge areas, material areas, and high-altitude platforms, combined with PTZ cameras and drones to form full-coverage monitoring. All high-definition cameras, PTZ cameras, and drones have a resolution of no less than 1920×1080 and a frame rate of 25-30 FPS to ensure clear and continuous images.

[0055] Image preprocessing: The acquired video stream is preprocessed as follows: distortion correction is performed based on multi-point calibration and OpenCV parameters; brightness equalization is achieved using the CLAHE algorithm; edge enhancement is achieved using the Sobel operator and sharpening method; and region cropping is performed to improve geometric accuracy and boundary awareness.

[0056] Keyframe Extraction and Output: To reduce computational redundancy, this embodiment employs a keyframe extraction algorithm based on frame difference, using a pixel change rate exceeding 5% as the keyframe extraction threshold. This approach ensures information integrity while filtering redundant frames. Based on the extracted keyframes, a structured image sequence is generated, with each frame in the sequence accompanied by a timestamp and location information, providing data support for subsequent region localization and multi-view image alignment.

[0057] S12. Parallel extraction of multi-dimensional risk features, the implementation methods of which include:

[0058] S21. Scene Semantic Segmentation and Risk Weight Mapping: This step is based on the image sequence obtained in step S1. It uses a deep learning semantic segmentation model to perform pixel-level parsing and outputs a semantic segmentation map containing semantic categories of the construction scene. A predefined regional importance weight value is mapped to each semantic category in the semantic segmentation map.

[0059] Figure 2 This embodiment demonstrates the network structure diagram of semantic segmentation and the visualization effect of semantic segmentation results in a typical construction scenario. Specifically, it includes the input original construction image, the semantic segmentation map output by the model (also known as the segmentation mask map), and the image overlay visualization effect for each semantic category, which can clearly identify the key elements of the construction scenario.

[0060] The deep learning semantic segmentation model is used to perform pixel-level parsing of construction images as input. The input consists of RGB images from an image sequence, all with a uniform size of 512×512. The output is a semantic segmentation map (Label Map) with the same size as the input, where each pixel value corresponds to its semantic category. These semantic categories include the function of the construction site area, personnel distribution, and equipment status. This deep learning semantic segmentation model is a pre-trained multi-scale semantic segmentation model based on a Transformer structure, preferably using SegFormer-B2 or DeepLabV3+ networks. These networks, leveraging their Transformer structure and multi-scale feature extraction capabilities, exhibit excellent segmentation performance for irregular boundaries and dense objects while maintaining computational efficiency. The processing is as follows: the encoder extracts multi-level features from the image, the decoder upsamples low-resolution features and restores the spatial distribution structure, and finally, the classification probability of each pixel is output through Softmax.

[0061] To enhance the model's generalization ability in construction scenarios, this embodiment uses a self-built construction semantic dataset as the training dataset during the training phase of the aforementioned deep learning semantic segmentation model. This dataset contains pixel-level annotations for 20 common construction targets, including high-altitude work areas, foundation pit edge areas, tower cranes, material storage areas, workers, exposed rebar, edge guardrails, and hot work areas. During training, data augmentation strategies such as random occlusion, mimicking rain / fog weather, or color perturbation are employed to expand the training set size, simulating real-world work environments under complex lighting and occlusion conditions. The training loss function uses a weighted combination of cross-entropy loss and IoU loss to balance boundary details and overall structure.

[0062] The module's output not only provides the location and shape information of each type of construction area, but also serves as the foundation for behavior recognition and hazard scoring. For example, the system's ability to identify the typical high-risk pattern of "personnel working in an area near an edge with no guardrails" relies on the spatial semantic information generated by this module.

[0063] S22. Personnel Behavior Recognition and Feature Extraction: This step uses target detection technology to locate construction workers in the image sequence and extracts key points of the human skeleton to construct skeletal features. Based on the skeletal features and semantic segmentation map, the behavioral risk probability of dangerous actions is calculated using an action discriminant function, thereby identifying unsafe actions of personnel in specific scenarios, such as not wearing safety belts, working outside designated areas, or working in overlapping areas. The skeletal features and behavioral risk probabilities constitute the personnel behavior features. In this embodiment, "specific scenario" refers to a specific semantic region or work context within the construction scene. It is a further refined expression of the construction scene, including the semantic region where the personnel are located, the current work environment, and the status of related equipment, rather than referring to the entire construction site in general.

[0064] Figure 3 This document demonstrates the workflow, including target localization, pose estimation, skeleton extraction, and behavior discrimination, and includes a typical illustration of a hazardous action recognition process. Specifically:

[0065] Using YOLOv5 as the basic detection framework, target-level positioning of construction personnel is performed, and the detection results are output as follows: Among them, Indicates the coordinates of the center point of the detection box. Width and height, For category confidence;

[0066] A lightweight pose estimation network is used to extract the skeletal keypoints for each construction worker, and the keypoint set is output. Where K is the number of key points. Let n be the coordinates of the nth key point. Indicates the confidence level of key points;

[0067] Construct skeleton vectors based on key point sets To obtain the dynamic features of the angle and length of the action changes, among which This represents the x-coordinate of the j-th keypoint in the image coordinate system. This represents the ordinate of the j-th keypoint in the image coordinate system; and the change in the angle between the keypoints is derived based on the aforementioned angular dynamic features. Based on the aforementioned dynamic length characteristics, the rate of change of skeleton segment length is derived. ,in, This represents the change in the angle between the corresponding skeleton vectors of adjacent frames. This represents the rate of change in the length of the corresponding skeleton segment between adjacent frames; to achieve the classification and recognition of illegal actions, this embodiment defines the action discrimination function as follows:

[0068] ;

[0069] in, This is the semantic scene mask for the current frame, generated by a deep learning semantic segmentation model; for , and The semantic segmentation model obtained through training;

[0070] The probability score of dangerous actions is calculated using an action discrimination function; if the probability score is greater than or equal to a preset threshold τ, a hazard marker is triggered and a risk assessment is performed.

[0071] To adapt to different operating conditions, this embodiment also designs a dynamic behavior template matching mechanism to identify risky behaviors. Specifically, the posture sequence composed of key points of the skeleton of the construction worker extracted in step S222 is embedded into a behavior template space pre-constructed by training with typical dangerous action samples, and the behavior is matched based on cosine similarity. Where A represents the current posture sequence feature vector of the construction worker, that is, the action features extracted from the skeletal key points of consecutive frames; B represents the template feature vector of a typical dangerous action in the template library, that is, the standard action template trained from historical dangerous behavior samples; calculate the similarity between the current posture sequence and the typical dangerous action template in the template library, and determine whether the current action belongs to the hidden risk behavior based on the similarity.

[0072] S23. Environmental risk factor extraction: Based on image sequences and / or external sensor data, identify the current construction environment status and output environmental risk factors;

[0073] This step is used to supplement the environmental risk factors required for subsequent risk assessment models. The extraction of environmental risk factors can be achieved based on existing mature technologies, such as: identifying weather conditions like rain, snow, fog, and insufficient sunlight through image analysis; acquiring environmental data such as temperature, humidity, wind speed, noise, or harmful gas concentrations through external sensors; or estimating on-site visibility using camera brightness information. This embodiment does not limit the specific types and acquisition methods of environmental risk factors; as long as they can reflect the impact of current construction conditions on personnel behavior and regional risks, they can be input as environmental risk factors into the subsequent risk assessment model.

[0074] S3. Multimodal feature fusion and preliminary hazard identification; for its implementation method, please refer to [link / reference]. Figure 4 This includes the following steps:

[0075] S31. Multimodal Feature Fusion: The purpose of this step is to combine information from different perception channels, including semantic features, human behavior features, device distribution, and spatial location, using feature-level alignment and channel-level weighting mechanisms, along with the self-attention mechanism of the Transformer structure, to construct a multimodal perception representation, i.e., fused features. Specifically:

[0076] The semantic segmentation graph from the semantic segmentation module The set of key points P in the skeleton features, and the camera position encoding. The timestamp encoding T of the current frame is uniformly projected into a high-dimensional vector space, and initially fused through vector concatenation to form an initial fused feature representation F:

[0077] ;

[0078] Here, ⊕ represents the vector concatenation operation. , , For each channel, the embedding function is... This indicates a vector concatenation operation.

[0079] It should be noted that in this step, "high-dimensional vector space" refers to a feature representation space of a unified dimension that the model can process, which is transformed from features from different sources, including semantic segmentation maps, skeletal keypoints, positional encodings, and temporal encodings, through an embedding function. The dimension can be set according to the model structure design, and is usually larger than the dimension of the original input features in order to accommodate richer semantic information. This embodiment does not limit the specific value of this dimension, and 128 dimensions, 256 dimensions, or other appropriate vector dimensions can be selected according to the specific implementation.

[0080] The initial fused feature representation F is input into the Transformer encoder. Through a self-attention mechanism, and by modeling the intrinsic dependencies and contextual relationships between different channels in feature F, the fused feature representation is output. .

[0081] To enhance the focus on key risk factors, namely the feature channels that contribute the most to the final hazard identification, this embodiment introduces a channel attention mechanism:

[0082] For fusion features An adaptive attention weight is calculated for each feature channel. The formula for calculating i is as follows:

[0083] ;

[0084] Among them, Indicates the first The weight vector of each channel, Score their attention.

[0085] Use the calculated attention weights Features Weighted summation is performed to obtain the final weighted global feature representation, which is the final fused feature. :

[0086] ;this It is a unified high-dimensional feature expression that integrates multi-source information and focuses on key risk factors.

[0087] S32. Hazard Identification:

[0088] Define the hazard detection function as follows:

[0089] ;

[0090] in It is the Sigmoid activation function. b are learnable parameters. The output value R(x) of the Sigmoid activation function is in the interval [0, 1], which represents the confidence that there is a hidden danger in the current scene.

[0091] The confidence level of a hazard's existence is calculated using a hazard discrimination function; when the confidence level of a hazard's existence exceeds a first preset threshold... If a potential hazard is identified, the process proceeds to the next step. The first preset threshold is set according to the type of construction scenario; in this embodiment, the first pre-screening threshold is 0.5.

[0092] S4. Comprehensive assessment of hazard levels. This step specifically includes:

[0093] To achieve efficient tiered response to hazards of varying severity, this embodiment comprehensively considers factors such as behavioral risk, the importance of the work area, and current environmental conditions, and designs a multi-factor weighted risk assessment model. The model calculates the total risk score S for each potential hazard, which serves as the core basis for determining the hazard level. Based on a second preset threshold, the total risk score is then used to classify the hazard levels.

[0094] The risk assessment model is defined as follows:

[0095] ;

[0096] in, This represents the risk probability score obtained in step S22. The region importance weight values ​​mapped in step S21, The output of step S23 represents environmental risk factors such as nighttime and severe weather. , , All of these are adjustable weight parameters.

[0097] To ensure the risk assessment model has good adaptability and accuracy under different construction scenarios, the values ​​of the weight parameters α, β, and γ in this embodiment are determined using a data-driven approach. The specific method is as follows:

[0098] Historical data samples from construction sites were collected to form a sample set containing hazard level labels; each sample in the sample set included a behavioral risk probability. Regional importance weight value Environmental risk factors And hazard level labels marked manually. The following objective function is constructed to solve for the weights:

[0099] Historical data samples from construction sites were collected to form a training sample set containing hazard level labels. Each sample includes a behavioral risk probability (Ract), a regional importance weight (Rarea), an environmental risk factor (Renv), and a corresponding manually labeled hazard level. For ease of description, the manually labeled hazard level is denoted as Ktrue. Based on this, an optimization objective function was constructed to solve for the weight parameters, making the risk score output by the model closer to the true risk level represented by Ktrue.

[0100] ;

[0101] in The objective is to minimize the error between the model's total risk score for potential hazards and the manual labeling.

[0102] After normalizing each factor, a grid search or gradient descent method is used to traverse the parameter space within the interval [0,1] to find the optimal parameter combination. To avoid overfitting of the risk assessment model, a cross-validation mechanism is introduced during the evaluation phase of the model to verify the effectiveness of parameter optimization; the optimal parameter combination selected through the above methods is then optimized. Once determined, it is incorporated into the final risk assessment model to meet the needs of real-time operational scenarios.

[0103] Regional importance weight value Based on the semantic segmentation map and preset mapping rules, the mapping rules are as follows: a predefined regional importance weight value is assigned to each semantic category, wherein the weight of high-altitude operation area and edge operation area is set to 1.0, the weight of hot work area and tower crane coverage area is set to 0.8, the weight of material stacking area and construction road is set to 0.5, and the weight of other low-risk areas is set to 0.2.

[0104] The total risk score S for each potential hazard is calculated using a risk assessment model; 0.45 and 0.75 are set as second preset thresholds, and the total risk score S is then classified according to these second preset thresholds to determine the hazard level K; the details are as follows:

[0105] ;

[0106] To facilitate regulators' intuitive assessment of hazard distribution, this embodiment generates a hazard heatmap based on a determined hazard level K. Different risk levels are mapped to the image space using a red-orange-yellow color gradient, and key areas are highlighted. This heatmap can be overlaid with the original image or semantic segmentation map to create a visualization effect.

[0107] Based on the determined hazard levels, this embodiment further constructs a multi-level early warning mechanism based on hazard response levels to achieve real-time alerts and closed-loop management of hazards at different risk levels. This mechanism aims to transform the algorithm's identification results into actionable management feedback, improving the safety response efficiency at construction sites. The specific implementation method is as follows:

[0108] The alarm response is triggered according to the level of the hazard: For Level I (serious) hazards, a multi-channel strong alert is triggered, including activation of the sound and light alarm, red flashing marker on the platform, and real-time push notification to the construction manager's mobile phone; for Level II (general) hazards, the alarm is presented through the management terminal in the form of visual icons combined with prompts and sounds; for Level III (minor) hazards, the alarm is only recorded in the background and presented during daily inspections.

[0109] Furthermore, the system supports integration with intelligent broadcasting systems and electrical control equipment, automatically issuing "work stoppage alerts" or initiating on-site broadcasts for high-risk hazards. All early warning events are accompanied by timestamps, image screenshots, and video traceability information, uploaded to the monitoring platform to support a closed-loop hazard handling process. To enhance human-computer interaction, risk area boundary boxes and behavior tags can be overlaid in real-time on the monitoring screen, helping on-site managers quickly locate problem areas and high-risk (Level I) hazard personnel.

[0110] An accident hazard investigation system based on scene recognition includes a multi-view data acquisition module, a multi-dimensional risk feature parallel extraction module, a multi-modal feature fusion and hazard preliminary judgment module, a hazard level comprehensive assessment module, and a graded response and early warning module. These modules are used to implement the aforementioned accident hazard investigation method based on scene recognition.

[0111] Deployment plan and field verification:

[0112] To verify the feasibility and effectiveness of the method of this invention in a real construction environment, the system was deployed and tested in a field project. The system adopts a three-layer architecture of "front-end acquisition + edge computing + central platform". The front end achieves multi-point deployment through high-definition cameras, wireless networks and power modules; the edge devices use NVIDIA Jetson or domestic computing power modules for real-time image analysis and hazard identification; the central server deploys a complete analysis platform and database for managing, recording and feedback all identification and early warning information.

[0113] The system supports integration with smart construction site management platforms, featuring a web-based monitoring interface, mobile app synchronization, and multi-user collaborative access management. Field tests show that in continuous monitoring scenarios, the system achieves an average frame rate of 15 FPS, a hazard identification accuracy rate of 91.6%, and an average early warning response time of less than 1 second for serious hazards, meeting the real-time and reliability requirements of construction sites.

[0114] Furthermore, based on the above method, this embodiment also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The characteristic of this embodiment is that the processor executes the computer program to implement the above-mentioned scene-based accident hazard investigation method. The algorithm used by the processor to execute the computer program to implement the above-mentioned scene-based accident hazard investigation method in this embodiment is as follows:

[0115]

Claims

1. A scene recognition-based accident hazard investigation method, characterized in that, The method comprises the following steps: S1. Image acquisition and processing: real-time acquisition of multi-view video streams of key areas of the construction site; image preprocessing and key frame extraction are performed on the video streams to generate a sequence of structure-standardized images; the image sequence is accompanied by timestamp and acquisition location information; S2. Parallel extraction of multi-dimensional risk features, specifically: S21. Scene semantic segmentation and risk weight mapping: based on the image sequence obtained in step S1, a deep learning semantic segmentation model is used for pixel-level analysis, and a semantic segmentation map containing the semantic categories of the construction scene is output; and each semantic category in the semantic segmentation map is mapped to a pre-defined regional importance weight value; S22. Personnel behavior recognition and feature extraction: target detection technology is used to locate the construction personnel in the image sequence, and human body pose estimation model is used to extract human body key points to construct skeleton features; according to the skeleton features and the semantic segmentation map, the behavior risk probability of dangerous actions is calculated by an action discrimination function, and the skeleton features and the behavior risk probability constitute the personnel behavior features; S23. Environment risk factor extraction: based on the image sequence and / or external sensor data, the current construction environment state is identified, and the environment risk factor is output; S3. Multi-modal feature fusion and preliminary hidden danger identification: the semantic segmentation map from step S21, the personnel behavior features from step S22, and the acquisition location information from step S1 are aligned and fused, the correlation between different features is modeled by a model based on self-attention mechanism, and the fused features are obtained; Based on the fused features, the hidden danger confidence is calculated by a hidden danger discrimination function; when the hidden danger confidence exceeds a first preset threshold, it is determined that there is a hidden danger and the subsequent step is entered; S4. Comprehensive assessment of hidden danger level: for the hidden danger identified in step S3, a risk assessment model is constructed based on the behavior risk probability, the regional importance weight value and the environment risk factor, the model calculates the weighted sum of each risk factor by a multi-factor weighting algorithm, and outputs a total risk score S of the hidden danger; According to the second preset threshold, the total risk score is divided into levels to determine the hidden danger level; S5. Graded response and early warning: according to the hidden danger level determined in step S4, a hidden danger heat map is generated and a corresponding multi-level early warning mechanism is triggered.

2. The method of claim 1, wherein: The deep learning semantic segmentation model in step S21 is a multi-scale semantic segmentation network based on SegFormer-B2 structure; the data set used to train the multi-scale semantic segmentation network contains pixel-level labels of at least 20 types of construction key targets, and data enhancement strategies such as random occlusion, color disturbance and weather simulation are used in the training.

3. The method of claim 1, wherein: In step S22, the action discrimination function is: ； Wherein P is a key point set in the skeleton feature, represents a key point angle change amount, is a skeleton segment length change rate, is a semantic segmentation map of the current frame, is a classification model trained.

4. The method of claim 1, wherein: The step S22 further comprises constructing a posture sequence of consecutive frames based on the key point set, and identifying implicit risk behaviors through a dynamic template matching mechanism; The dynamic template matching mechanism is realized by calculating the cosine similarity between the posture sequence and the pre-stored dangerous action template.

5. The method of claim 1, wherein: In step S3, the specific method of feature alignment and fusion is: The semantic segmentation map, the personnel behavior feature and the collection position information are spliced after being mapped to a high-dimensional space through an embedding function, a self-attention mechanism of a Transformer encoder is used to model the dependency relationship between the features, and a channel attention mechanism is used for weighting to obtain the fusion features .

6. The method of claim 1, wherein: In step S4, the risk assessment model is: ； wherein, represents the behavior risk probability obtained in step S22; is the region importance weight value mapped in step S21; , , are all adjustable weight parameters, in [0, 1] or a corresponding interval according to the actual situation.

7. The method of claim 6, wherein: The step S4 grades the total risk score S according to a second preset threshold to determine the hazard level K, and the specific process is as follows: 。 8. The method of claim 1, wherein The multi-level early warning mechanism in the step S5 includes: For Level hidden danger trigger sound and light alarm and multi-terminal real-time push; For Level hidden danger trigger management terminal visual alarm and work order; For Level hidden trouble generation background inspection record.

9. An accident hazard investigation system based on scene recognition, comprising a multi-view data acquisition module, a multi-dimensional risk feature parallel extraction module, a multi-modal feature fusion and hazard preliminary judgment module, a hazard level comprehensive evaluation module and a hierarchical response and early warning, which realize the accident hazard investigation method based on scene recognition as claimed in any one of claims 1 to 8 through the modules.

Images