A highway concrete guardrail construction site safety control system
By introducing video acquisition and behavioral feature modeling technology at the construction site of highway concrete guardrails, the problems of difficulty in quantifying modeling and continuously tracking the behavior of construction personnel and equipment in existing technologies have been solved. This has enabled precise location of abnormal behavior and attribution of responsibility, and improved the intelligence and precision of safety management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JIANGXI HIGHWAY MANAGEMENT BUREAU TRAFFIC ENG CO
- Filing Date
- 2026-02-25
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies make it difficult to quantitatively model and continuously track the continuous movement trajectories of construction personnel and machinery at highway concrete guardrail construction sites. They cannot distinguish between normal high-risk operations and abnormal dangerous behaviors under complex working conditions, and cannot automatically associate detected abnormal behaviors with construction plans and safety management systems, resulting in delayed response and difficulties in individualized risk assessment.
The system uses a video acquisition module to acquire image frame sequences, a target detection and tracking module to extract behavioral feature sequences, and a behavior fingerprint modeling module to perform joint modeling. It also uses an anomaly window judgment module and a semantic responsibility judgment module to generate security control instructions, thereby achieving accurate location of abnormal behavior and attribution of responsibility.
It enables individualized risk assessment and dynamic operation permission control for construction personnel and machinery, significantly improving the ability to identify hidden dangerous behaviors and the level of precision and intelligence in safety management, and can generate safety behavior files and operation permission control instructions.
Smart Images
Figure CN122157095A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of highway safety management and control technology, and in particular to a safety management and control system for highway concrete guardrail construction sites. Background Technology
[0002] During the construction of concrete guardrails on highways, on-site safety management usually relies on safety officer patrols, playback of video recordings from fixed monitoring cameras, and simple alarm devices. Existing video monitoring systems are mostly based on manual visual inspection or target detection based on single-frame images. They can only roughly identify obvious violations such as people entering restricted areas or not wearing protective equipment. It is difficult to quantitatively model and continuously track the continuous movement trajectory and operation mode of construction personnel and machinery over a period of time, and it is impossible to form behavioral characteristic files that can be used for long-term evaluation.
[0003] Furthermore, existing intelligent analysis methods mostly use threshold triggering or rule matching to issue alarms, lacking in-depth modeling of temporal behavioral characteristics. They are unable to distinguish between normal high-risk operations under complex working conditions and truly abnormal and dangerous behaviors. They also cannot automatically associate detected abnormal behaviors with construction plans, safety technical briefings, and safety management system clauses, making it impossible to identify the responsible parties and corresponding clauses. This results in on-site handling and subsequent management relying on human experience judgment, leading to shortcomings such as delayed response, coarse control granularity, and difficulty in achieving individualized risk assessment and dynamic control of work permissions for personnel and equipment. Summary of the Invention
[0004] To address the technical problems existing in the background art, this invention proposes a safety management and control system for highway concrete guardrail construction sites.
[0005] This invention proposes a safety management system for highway concrete guardrail construction sites, comprising: The video acquisition module is used to acquire image frame sequences; The target detection and tracking module is used to process each image frame in the image frame sequence to obtain the bounding box of the candidate target, and to obtain the behavioral feature sequence of the candidate target based on the bounding box; The behavior fingerprint modeling module is used to extract image segment sequences based on behavior feature sequences, and to jointly model the behavior feature sequences and image segment sequences to obtain the anomaly score of each candidate target at each time step and the behavior fingerprint vector within the current monitoring period. The anomaly window determination module is used to perform statistical analysis on the anomaly scores using a sliding time window to obtain anomaly windows, and to construct multimodal samples of abnormal behavior based on each anomaly window; The semantic accountability module is used to obtain semantic accountability results based on multimodal samples of abnormal behavior; The security control instruction module is used to generate security control instruction vectors based on semantic judgment results.
[0006] Preferably, the image frame sequence is obtained as follows: Obtain a plan of the construction area of the highway concrete guardrail, delineate the target monitoring area on the plan, and determine the installation coordinates of the video acquisition device within the target monitoring area; For each video acquisition device, the installation height, pitch angle, and horizontal azimuth angle are set, and the internal parameters are calibrated, including focal length, principal point coordinates, and distortion coefficient. Video data is continuously acquired at a fixed frame rate using various video acquisition devices to obtain an image frame sequence, where each image frame in the image frame sequence is a two-dimensional pixel matrix.
[0007] Preferably, each image frame in the image frame sequence is processed to obtain the bounding box of the candidate target, and the behavioral feature sequence of the candidate target is obtained based on the bounding box, as follows: Each image frame in the image frame sequence is input into the YOLOv5-based object detection network for processing, resulting in a set of bounding boxes for all candidate targets corresponding to the image frame. The coordinates of the bounding box corresponding to each candidate target in the bounding box set are composed of the x-coordinate of the top left corner pixel, the y-coordinate of the top left corner pixel, the x-coordinate of the bottom right corner pixel, and the y-coordinate of the bottom right corner pixel. Calculate the center pixel coordinates of the bounding box for each candidate target: ; ; in, Let x be the x-coordinate of the top-left pixel of the bounding box of the j-th candidate target in the k-th image frame of the i-th video. The x-coordinate of the bottom right corner of the bounding box. The ordinate of the top-left corner of the bounding box (in pixels). The y-coordinate of the bottom right corner of the bounding box. The x-coordinate of the center pixel. The ordinate of the center pixel; Based on the installation location coordinates, installation height, pitch angle, horizontal azimuth angle, and intrinsic parameters of the video acquisition device, a homography matrix corresponding to the video acquisition device is constructed. A coordinate transformation is then performed on the center pixel coordinates to obtain the spatial location coordinates on the construction plane. : ; in, This is the scaling factor for the corresponding coordinate transformation. Let x be the x-coordinate of the candidate target in the construction plane coordinate system. The corresponding ordinate, It is a homography matrix; Convert spatial coordinates into position vectors And match the position vector with the corresponding image frame timestamp and video capture device identification Associate them to form triples ; Data association is performed on the triples of candidate targets detected in image frames at adjacent time points, and a target identifier is assigned to each candidate target. Triples belonging to the same target identifier are arranged in chronological order to form a temporal position sequence of candidate targets. ,in, For candidate targets at time The corresponding position vector, The length of the temporal location sequence of the candidate target; For each candidate target at adjacent time points and Calculate the velocity vector: ; in, Indicates the horizontal velocity. Indicates vertical velocity; And based on the velocity vector, construct a behavior feature vector for each candidate target at each time t. The behavioral feature vectors of the same candidate target at different times are combined to obtain the behavioral feature sequence of the candidate target.
[0008] Preferably, image segment sequences are extracted based on behavioral feature sequences, and the behavioral feature sequences and image segment sequences are jointly modeled to obtain the anomaly score of each candidate target at each time step and the behavioral fingerprint vector within the current monitoring period, as follows: Extract the position vector of the candidate frame corresponding to the behavioral feature sequence. The image segment sequence is obtained and input into the encoding unit of ViT. The position vector of each candidate frame in the image segment sequence is encoded to obtain the corresponding visual feature vector. The visual feature vector and behavioral feature vector at each time step are concatenated to construct a fused feature vector. ; fuse feature vectors Input the SRU++ temporal modeling unit, perform temporal modeling on the fused feature vector at each time step according to the time sequence, construct the state update relationship of the SRU++ unit, and obtain the hidden state vectors of the same candidate target at different time steps. The SRU++ model, based on SRU, further introduces an attention mechanism, enabling the model to capture key information more effectively when processing long sequence data. ; ; ; ; ; in, Forget gate vector, To reset the gate vector, For candidate state vectors, For at any time The unit state vector, This is the unit state vector from the previous time step. For at any time The hidden state vector, , , This is the corresponding weight matrix. , , For the corresponding bias vector, For the Sigmoid function, It is a non-linear activation function. This is an element-wise multiplication operation; Hidden state vectors of the same candidate target at different times Perform convergence operations to obtain the behavioral fingerprint vectors of candidate targets. : ; in, Indicates the length of the temporal position sequence of the candidate target; Based on hidden state vector Constructing Reconstructed Vectors : ; in, This is the output layer weight matrix. This is the output layer bias vector; Based on the fusion feature vector and reconstructed vector The anomaly score of each candidate target at each time step is obtained by calculating the L2 norm. : ; in, It is a 2-norm.
[0009] Preferably, the ViT encoding unit and the SRU++ time-series modeling unit are trained by selecting behavioral feature sequences of several candidate targets obtained during historical normal construction processes as training samples, and the reconstruction error of all training samples is used as the loss function: ; in, It is a 2-norm. This represents the total number of training samples, used to optimize the training of parameters for the visual Transformer encoding unit and the SRU++ temporal modeling unit.
[0010] Preferably, anomaly windows are obtained by statistical analysis of anomaly scores using a sliding time window, and multimodal samples of anomalous behavior are constructed based on each anomaly window, as follows: For the target identifier is For candidate targets, the abnormal scores of each time point within the current monitoring period are combined into an abnormal score sequence. ; The abnormal score sequence is divided into sliding windows, and the average abnormal score within each window is calculated. and the maximum abnormal score within the window , Indicates the first One window; Set the average anomaly score threshold and the maximum anomaly score threshold The average anomaly score threshold and the maximum anomaly score threshold are set based on expert experience. Anomaly detection is performed for each window, and the threshold is set when the average anomaly score within the window is reached. Less than the average anomaly score threshold And the maximum abnormal score within the window Less than the maximum anomaly score threshold If the window is in a normal state, it is considered a normal window; otherwise, it is considered an abnormal window. For each abnormal window, acquire all image frames between the start time and the end time to form the original video clip set. ; For each anomaly score sequence within an anomaly window, the index of the time with the highest anomaly score is selected, and the target region image corresponding to the index of the time with the highest anomaly score is extracted from the image frame sequence within the current monitoring period as the keyframe image. ; Based on the abnormal window, retrieve the corresponding set of construction process information from the construction process information. Search the construction plan text text for the set of construction plan text fragments corresponding to the abnormal window. Search the set of clauses in the safety management system that correspond to the exception window. ; The original video clip set, keyframe images, construction procedure information set, construction plan text fragment set, and clause text set are combined to obtain the abnormal behavior multimodal sample corresponding to each abnormal window. .
[0011] Preferably, semantic responsibility determination results are obtained from multimodal samples of abnormal behavior, as follows: Based on the multimodal samples of abnormal behavior, keyframe images, sets of construction process information, sets of construction plan text fragments, and sets of clause texts are selected as semantic responsibility inputs; The keyframe images are input into the visual encoder for feature extraction to obtain the visual encoding vector. ; The set of construction procedure information, the set of construction plan text fragments, and the set of clause texts are respectively input into the text encoder to obtain the feature vector of construction procedure information. Construction plan text feature vector and the feature vector of the clause text ; The visual encoding vector is concatenated with the feature vectors of construction process information, construction plan text, and clause text to construct a multimodal fusion feature vector. ; The multimodal fusion feature vector is input into the visual language semantic responsibility determination network. The probability distribution of abnormal behavior categories is obtained through the abnormal behavior category determination sub-network, the probability distribution of responsible entity categories is obtained through the responsible entity identification determination sub-network, the probability distribution of clause relevance is obtained through the safety management clause identification determination sub-network, and the probability distribution of handling suggestion categories is obtained through the handling suggestion determination sub-network. Based on the probability distributions of abnormal behavior categories, responsible entity categories, clause relevance, and handling suggestion categories, the abnormal behavior category identifier is obtained by selecting the index corresponding to the highest probability. Responsible entity identification Safety Management Clause Identification Set and disposal suggestion labels Obtain semantic accountability results for multimodal samples of abnormal behavior. .
[0012] Preferably, the visual language semantic responsibility determination network consists of an abnormal behavior category determination subnetwork, a responsible entity identification determination subnetwork, a safety management clause identification determination subnetwork, and a handling suggestion determination subnetwork. Each subnetwork consists of an independent linear transformation layer and an output layer. Each output layer outputs the predicted probability distribution through a sigmoid function.
[0013] Preferably, a security control instruction vector is generated based on the semantic accountability result, as follows: For each candidate target, based on the behavioral fingerprint vector of the current monitoring period... And the obtained historical normal construction process behavior fingerprint vector Calculate behavioral fingerprint deviation ; ; in, It is a 2-norm; Get the number of abnormal windows of candidate targets Based on anomaly score sequences Calculate the average anomaly score for the current monitoring period. : ; A comprehensive risk index for candidate targets is obtained based on behavioral fingerprint deviation, the number of abnormal windows, and the average abnormal score during the current monitoring period. : ; in, Indicates the total number of windows. , , These are the weighting coefficients; The semantic judgment results corresponding to each abnormal window The aggregation process yields a set of semantic accountability results for candidate targets within the current monitoring period. Based on the semantic accountability result set, a security behavior profile record vector is constructed for each candidate target: ; in, For target identification; Risk level identification is determined based on comprehensive risk indicators. The risk level identifiers include low risk, medium risk, and high risk: ; in, Indicates a low-risk level. Indicates a medium-risk level. Indicates a high-risk level; Based on the risk level identifier and the semantic assessment result set, generate a safety control instruction vector corresponding to the candidate target: ; in, This is a job access control identifier, with a value of [value]. , This indicates that normal operation is permitted. Indicates operation under restrictive conditions. This indicates that work is prohibited. This is an alarm level identifier used to indicate the level of alarm output. An update identifier for safety management records is used to indicate whether the current semantic accountability information should be written into the safety management record of the highway concrete guardrail construction site, and to associate and store the safety control instruction vector with the safety behavior archive record vector.
[0014] The safety management system for highway concrete guardrail construction sites proposed in this invention has the following beneficial technical effects: 1. This application introduces a ViT and SRU++-based temporal modeling method at the construction site to construct behavioral feature sequences based on the temporal position, speed, and other behavioral characteristics of construction personnel and machinery. This enables the output of behavioral fingerprint vectors and anomaly scores, achieving dynamic behavioral pattern modeling and quantitative description of individuals within a complete work cycle. By combining real-time anomaly score calculation and anomaly window determination using a sliding time window, this application can accurately pinpoint the time interval of abnormal behavior and corresponding candidate targets during the construction process, significantly improving the ability to identify hidden and progressive dangerous behaviors. Compared with existing technologies that rely solely on single-frame detection or simple threshold rules, this application shows significant improvements in continuity, sensitivity, and reliability.
[0015] 2. Based on abnormal behavior detection, this application further constructs multimodal samples of abnormal behavior from keyframe images, construction process information, construction plan text, and safety management system clause text. It utilizes a visual encoder and a text encoder to fuse features, building a visual-language semantic accountability model for construction safety scenarios. This model directly outputs the abnormal behavior category, responsible entity identifier, corresponding safety management clause identifier set, and handling suggestion identifier. Furthermore, it combines the deviation between historical and current behavior fingerprints, average abnormal score, and abnormal window ratio to form a comprehensive risk index and risk level. This generates safety behavior profiles and work permission control instructions bound to candidate targets. This not only achieves a closed-loop accountability process from "detecting abnormal behavior" to "automatically providing clause basis and responsibility attribution," but also enables individualized risk grading and dynamic work permission control for construction personnel and machinery. It overcomes the shortcomings of existing technologies, such as the difficulty in accurately correlating video monitoring results with safety system clauses and the difficulty in quantifying long-term individual safety performance, significantly improving the refinement and intelligence of safety management at highway concrete guardrail construction sites. Attached Figure Description
[0016] Figure 1 This is a schematic diagram of a safety management and control system for highway concrete guardrail construction sites according to the present invention. Detailed Implementation
[0017] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.
[0018] like Figure 1 The safety management system for highway concrete guardrail construction site shown includes the following steps: The video acquisition module is used to acquire image frame sequences; In an optional embodiment, the image frame sequence is obtained as follows: Obtain a plan of the construction area of the highway concrete guardrail, delineate the target monitoring area on the plan, and determine the installation coordinates of the video acquisition device within the target monitoring area; For each video acquisition device, the installation height, pitch angle, and horizontal azimuth angle are set, and the internal parameters are calibrated, including focal length, principal point coordinates, and distortion coefficient. Video data is continuously acquired at a fixed frame rate using various video acquisition devices to obtain an image frame sequence, where each image frame in the image frame sequence is a two-dimensional pixel matrix.
[0019] The target detection and tracking module is used to process each image frame in the image frame sequence to obtain the bounding box of the candidate target, and to obtain the behavioral feature sequence of the candidate target based on the bounding box; In an optional embodiment, each image frame in the image frame sequence is processed to obtain the bounding box of the candidate target, and the behavioral feature sequence of the candidate target is obtained based on the bounding box, as follows: Each image frame in the image frame sequence is input into the YOLOv5-based object detection network for processing, resulting in a set of bounding boxes for all candidate targets corresponding to the image frame. The coordinates of the bounding box corresponding to each candidate target in the bounding box set are composed of the x-coordinate of the top left corner pixel, the y-coordinate of the top left corner pixel, the x-coordinate of the bottom right corner pixel, and the y-coordinate of the bottom right corner pixel. Calculate the center pixel coordinates of the bounding box for each candidate target: ; ; in, Let x be the x-coordinate of the top-left pixel of the bounding box of the j-th candidate target in the k-th image frame of the i-th video. The x-coordinate of the bottom right corner of the bounding box. The ordinate of the top-left corner of the bounding box (in pixels). The y-coordinate of the bottom right corner of the bounding box. The x-coordinate of the center pixel. The ordinate of the center pixel; Based on the installation location coordinates, installation height, pitch angle, horizontal azimuth angle, and intrinsic parameters of the video acquisition device, a homography matrix corresponding to the video acquisition device is constructed. A coordinate transformation is then performed on the center pixel coordinates to obtain the spatial location coordinates on the construction plane. : ; in, This is the scaling factor for the corresponding coordinate transformation. Let x be the x-coordinate of the candidate target in the construction plane coordinate system. The corresponding ordinate, It is a homography matrix; Convert spatial coordinates into position vectors And match the position vector with the corresponding image frame timestamp and video capture device identification Associate them to form triples ; Data association is performed on the triples of candidate targets detected in image frames at adjacent time points, and a target identifier is assigned to each candidate target. Triples belonging to the same target identifier are arranged in chronological order to form a temporal position sequence of candidate targets. ,in, For candidate targets at time The corresponding position vector, The length of the temporal location sequence of the candidate target; For each candidate target at adjacent time points and Calculate the velocity vector: ; in, Indicates the horizontal velocity. Indicates vertical velocity; And based on the velocity vector, construct a behavior feature vector for each candidate target at each time t. The behavioral feature vectors of the same candidate target at different times are combined to obtain the behavioral feature sequence of the candidate target.
[0020] The behavior fingerprint modeling module is used to extract image segment sequences based on behavior feature sequences, and to jointly model the behavior feature sequences and image segment sequences to obtain the anomaly score of each candidate target at each time step and the behavior fingerprint vector within the current monitoring period. In an optional embodiment, an image segment sequence is extracted based on the behavioral feature sequence. The behavioral feature sequence and the image segment sequence are jointly modeled to obtain the anomaly score of each candidate target at each time step and the behavioral fingerprint vector within the current monitoring period, as follows: Extract the position vector of the candidate frame corresponding to the behavioral feature sequence. The image segment sequence is obtained and input into the encoding unit of ViT. The position vector of each candidate frame in the image segment sequence is encoded to obtain the corresponding visual feature vector. The visual feature vector and behavioral feature vector at each time step are concatenated to construct a fused feature vector. ; fuse feature vectors Input the SRU++ temporal modeling unit, perform temporal modeling on the fused feature vector at each time step according to the time sequence, construct the state update relationship of the SRU++ unit, and obtain the hidden state vectors of the same candidate target at different time steps. The SRU++ model, based on SRU, further introduces an attention mechanism, enabling the model to capture key information more effectively when processing long sequence data. ; ; ; ; ; in, Forget gate vector, To reset the gate vector, For candidate state vectors, For at any time The unit state vector, This is the unit state vector from the previous time step. For at any time The hidden state vector, , , This is the corresponding weight matrix. , , For the corresponding bias vector, For the Sigmoid function, It is a non-linear activation function. This is an element-wise multiplication operation; Hidden state vectors of the same candidate target at different times Perform convergence operations to obtain the behavioral fingerprint vectors of candidate targets. : ; in, Indicates the length of the temporal position sequence of the candidate target; Based on hidden state vector Constructing Reconstructed Vectors : ; in, This is the output layer weight matrix. This is the output layer bias vector; Based on the fusion feature vector and reconstructed vector The anomaly score of each candidate target at each time step is obtained by calculating the L2 norm. : ; in, It is a 2-norm; In an optional embodiment, the ViT encoding unit and the SRU++ temporal modeling unit are trained by selecting behavioral feature sequences of several candidate targets obtained during historical normal construction processes as training samples, and the reconstruction error of all training samples is used as the loss function: ; in, It is a 2-norm. This represents the total number of training samples, used to optimize the training of parameters for the visual Transformer encoding unit and the SRU++ temporal modeling unit.
[0021] The anomaly window determination module is used to perform statistical analysis on the anomaly scores using a sliding time window to obtain anomaly windows, and to construct multimodal samples of abnormal behavior based on each anomaly window; In an optional embodiment, anomaly windows are obtained by statistical analysis of anomaly scores using a sliding time window, and multimodal samples of anomalous behavior are constructed based on each anomaly window, as follows: For the target identifier is For candidate targets, the abnormal scores of each time point within the current monitoring period are combined into an abnormal score sequence. ; The abnormal score sequence is divided into sliding windows, and the average abnormal score within each window is calculated. and the maximum abnormal score within the window , Indicates the first One window; Set the average anomaly score threshold and the maximum anomaly score threshold The average anomaly score threshold and the maximum anomaly score threshold are set based on expert experience. Anomaly detection is performed for each window, and the threshold is set when the average anomaly score within the window is reached. Less than the average anomaly score threshold And the maximum abnormal score within the window Less than the maximum anomaly score threshold If the window is in a normal state, it is considered a normal window; otherwise, it is considered an abnormal window. For each abnormal window, acquire all image frames between the start time and the end time to form the original video clip set. ; For each anomaly score sequence within an anomaly window, the index of the time with the highest anomaly score is selected, and the target region image corresponding to the index of the time with the highest anomaly score is extracted from the image frame sequence within the current monitoring period as the keyframe image. ; Based on the abnormal window, retrieve the corresponding set of construction process information from the construction process information. Search the construction plan text text for the set of construction plan text fragments corresponding to the abnormal window. Search the set of clauses in the safety management system that correspond to the exception window. ; The original video clip set, keyframe images, construction procedure information set, construction plan text fragment set, and clause text set are combined to obtain the abnormal behavior multimodal sample corresponding to each abnormal window. .
[0022] The semantic accountability module is used to obtain semantic accountability results based on multimodal samples of abnormal behavior; In an optional embodiment, semantic responsibility determination results are obtained from multimodal samples of abnormal behavior, as follows: Based on the multimodal samples of abnormal behavior, keyframe images, sets of construction process information, sets of construction plan text fragments, and sets of clause texts are selected as semantic responsibility inputs; The keyframe images are input into the visual encoder for feature extraction to obtain the visual encoding vector. ; The set of construction procedure information, the set of construction plan text fragments, and the set of clause texts are respectively input into the text encoder to obtain the feature vector of construction procedure information. Construction plan text feature vector and the feature vector of the clause text ; The visual encoding vector is concatenated with the feature vectors of construction process information, construction plan text, and clause text to construct a multimodal fusion feature vector. ; The multimodal fusion feature vector is input into the visual language semantic responsibility determination network. The probability distribution of abnormal behavior categories is obtained through the abnormal behavior category determination sub-network, the probability distribution of responsible entity categories is obtained through the responsible entity identification determination sub-network, the probability distribution of clause relevance is obtained through the safety management clause identification determination sub-network, and the probability distribution of handling suggestion categories is obtained through the handling suggestion determination sub-network. Based on the probability distributions of abnormal behavior categories, responsible entity categories, clause relevance, and handling suggestion categories, the abnormal behavior category identifier is obtained by selecting the index corresponding to the highest probability. Responsible entity identification Safety Management Clause Identification Set and disposal suggestion labels Obtain semantic accountability results for multimodal samples of abnormal behavior. ; In an optional embodiment, the visual language semantic responsibility determination network consists of an abnormal behavior category determination subnetwork, a responsible entity identification determination subnetwork, a safety management clause identification determination subnetwork, and a handling suggestion determination subnetwork. Each subnetwork consists of an independent linear transformation layer and an output layer. Each output layer outputs the predicted probability distribution through a sigmoid function.
[0023] The security control instruction module is used to generate security control instruction vectors based on semantic judgment results. In an optional embodiment, a security control instruction vector is generated based on the semantic accountability result, as follows: For each candidate target, based on the behavioral fingerprint vector of the current monitoring period... And the obtained historical normal construction process behavior fingerprint vector Calculate behavioral fingerprint deviation ; ; in, It is a 2-norm; Get the number of abnormal windows of candidate targets Based on anomaly score sequences Calculate the average anomaly score for the current monitoring period. : ; A comprehensive risk index for candidate targets is obtained based on behavioral fingerprint deviation, the number of abnormal windows, and the average abnormal score during the current monitoring period. : ; in, Indicates the total number of windows. , , These are the weighting coefficients; The semantic judgment results corresponding to each abnormal window The aggregation process yields a set of semantic accountability results for candidate targets within the current monitoring period. Based on the semantic accountability result set, a security behavior profile record vector is constructed for each candidate target: ; in, For target identification; Risk level identification is determined based on comprehensive risk indicators. The risk level identifiers include low risk, medium risk, and high risk: ; in, Indicates a low-risk level. Indicates a medium-risk level. Indicates a high-risk level; Based on the risk level identifier and the semantic assessment result set, generate a safety control instruction vector corresponding to the candidate target: ; in, This is a job access control identifier, with a value of [value]. , This indicates that normal operation is permitted. Indicates operation under restrictive conditions. This indicates that work is prohibited. This is an alarm level identifier used to indicate the level of alarm output. An update identifier for safety management records is used to indicate whether the current semantic accountability information should be written into the safety management record of the highway concrete guardrail construction site, and to associate and store the safety control instruction vector with the safety behavior archive record vector.
[0024] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A safety management and control system for highway concrete guardrail construction sites, characterized in that, include: The video acquisition module is used to acquire image frame sequences; The target detection and tracking module is used to process each image frame in the image frame sequence to obtain the bounding box of the candidate target, and to obtain the behavioral feature sequence of the candidate target based on the bounding box; The behavior fingerprint modeling module is used to extract image segment sequences based on behavior feature sequences, and to jointly model the behavior feature sequences and image segment sequences to obtain the anomaly score of each candidate target at each time step and the behavior fingerprint vector within the current monitoring period. The anomaly window determination module is used to perform statistical analysis on the anomaly scores using a sliding time window to obtain anomaly windows, and to construct multimodal samples of abnormal behavior based on each anomaly window; The semantic accountability module is used to obtain semantic accountability results based on multimodal samples of abnormal behavior; The security control instruction module is used to generate security control instruction vectors based on semantic judgment results.
2. The safety management and control system for highway concrete guardrail construction site according to claim 1, characterized in that, The image frame sequence is obtained as follows: Obtain a plan of the construction area of the highway concrete guardrail, delineate the target monitoring area on the plan, and determine the installation coordinates of the video acquisition device within the target monitoring area; For each video acquisition device, the installation height, pitch angle, and horizontal azimuth angle are set, and the internal parameters are calibrated, including focal length, principal point coordinates, and distortion coefficient. Video data is continuously acquired at a fixed frame rate using various video acquisition devices to obtain an image frame sequence.
3. The safety management and control system for highway concrete guardrail construction site according to claim 1, characterized in that, Each image frame in the image frame sequence is processed to obtain the bounding box of the candidate target. Based on the bounding box, the behavioral feature sequence of the candidate target is obtained, as follows: Each image frame in the image frame sequence is input into the YOLOv5-based object detection network for processing, resulting in a set of bounding boxes for all candidate targets corresponding to the image frame. The coordinates of the bounding box corresponding to each candidate target in the bounding box set are composed of the x-coordinate of the top left corner pixel, the y-coordinate of the top left corner pixel, the x-coordinate of the bottom right corner pixel, and the y-coordinate of the bottom right corner pixel. Calculate the center pixel coordinates of the bounding box for each candidate target; Based on the installation location coordinates, installation height, pitch angle, horizontal azimuth angle and intrinsic parameters of the video acquisition device, construct the homography matrix corresponding to the video acquisition device, and perform coordinate transformation on the center pixel coordinates to obtain the spatial position coordinates on the construction plane. The spatial coordinates are converted into position vectors, and the position vectors are associated with the corresponding image frame timestamps and video acquisition device identifiers to form a triplet; Data association is performed on the triples of candidate targets detected in image frames at adjacent time points. A target identifier is assigned to each candidate target, and triples belonging to the same target identifier are arranged in chronological order to form a temporal position sequence of candidate targets. For each candidate target, calculate the velocity vector at adjacent time points, and construct a behavior feature vector for each candidate target at each time point based on the velocity vector. Combine the behavior feature vectors of the same candidate target at different time points to obtain the behavior feature sequence of the candidate targets.
4. The safety management and control system for highway concrete guardrail construction site according to claim 1, characterized in that, Image segment sequences are extracted based on behavioral feature sequences. A joint model is then performed on the behavioral feature sequences and image segment sequences to obtain the anomaly score for each candidate target at each time step and its behavioral fingerprint vector within the current monitoring period, as follows: The position vectors of candidate frames corresponding to the behavioral feature sequence are extracted to obtain the image segment sequence. The image segment sequence is input into the encoding unit of ViT to encode the position vector of each candidate frame in the image segment sequence to obtain the corresponding visual feature vector. The visual feature vector and behavioral feature vector at each moment are concatenated to construct a fused feature vector; The fused feature vector is input into the SRU++ temporal modeling unit to obtain the hidden state vectors of the same candidate target at different times. By performing a convergence operation on the hidden state vectors of the same candidate target at different times, the behavioral fingerprint vector of the candidate target is obtained. A reconstruction vector is constructed based on the hidden state vector. The abnormal score of each candidate target at each time step is obtained by calculating the L2 norm based on the fused feature vector and the reconstruction vector.
5. A safety management and control system for highway concrete guardrail construction sites according to claim 4, characterized in that, The ViT encoding unit and the SRU++ time-series modeling unit are trained by selecting behavioral feature sequences of several candidate targets obtained during historical normal construction processes as training samples, and the reconstruction error of all training samples is used as the loss function.
6. A safety management and control system for highway concrete guardrail construction sites according to claim 1, characterized in that, Anomaly windows were obtained by statistical analysis of anomaly scores using a sliding time window, and multimodal samples of anomalous behavior were constructed based on each anomaly window, as follows: The abnormal scores of the candidate targets at each moment within the current monitoring period are combined into an abnormal score sequence; The abnormal score sequence is divided into sliding windows, and the average abnormal score and the maximum abnormal score within each window are calculated. Set an average abnormal score threshold and a maximum abnormal score threshold, and determine the abnormality of each window. When the average abnormal score in the window is less than the average abnormal score threshold and the maximum abnormal score in the window is less than the maximum abnormal score threshold, the window is determined to be a non-abnormal window; otherwise, the window is determined to be an abnormal window. For each abnormal window, obtain all image frames between the start time and the end time to form a set of original video clips; For each abnormal score sequence within an abnormal window, select the index of the time with the maximum abnormal score, and extract the target area image corresponding to the index of the time with the maximum abnormal score from the image frame sequence within the current monitoring period as the key frame image. Based on the abnormal window, the corresponding set of construction process information is read from the construction process information, the set of construction plan text fragments corresponding to the abnormal window is retrieved from the construction plan text, and the set of clause texts corresponding to the abnormal window is retrieved from the safety management system clause text. The original video clip set, keyframe images, construction procedure information set, construction plan text fragment set, and clause text set are combined to obtain the abnormal behavior multimodal sample corresponding to each abnormal window.
7. A safety management and control system for highway concrete guardrail construction sites according to claim 1, characterized in that, The semantic responsibility determination results are obtained from multimodal samples of abnormal behavior, as follows: Based on the multimodal samples of abnormal behavior, keyframe images, sets of construction process information, sets of construction plan text fragments, and sets of clause texts are selected as semantic responsibility inputs; The keyframe images are input into the visual encoder for feature extraction to obtain the visual encoding vector; Input the set of construction procedure information, the set of construction plan text fragments, and the set of clause texts into the text encoder to obtain the feature vectors of construction procedure information, construction plan texts, and clause texts, respectively. The visual encoding vector is concatenated with the feature vectors of construction process information, construction plan text, and clause text to construct a multimodal fusion feature vector; The multimodal fusion feature vectors are input into the visual language semantic accountability network to obtain the probability distribution of abnormal behavior categories, the probability distribution of responsible subject categories, the probability distribution of clause relevance, and the probability distribution of disposal suggestion categories. Based on the probability distributions of abnormal behavior categories, responsible entity categories, clause relevance, and handling suggestion categories, the set of abnormal behavior category identifiers, responsible entity identifiers, safety management clause identifiers, and handling suggestion identifiers are obtained by selecting the index corresponding to the highest probability, thus obtaining the semantic responsibility judgment results of the multimodal abnormal behavior samples.
8. A safety management and control system for highway concrete guardrail construction sites according to claim 7, characterized in that, The visual language semantic responsibility determination network consists of an abnormal behavior category determination subnetwork, a responsible entity identification determination subnetwork, a safety management clause identification determination subnetwork, and a handling suggestion determination subnetwork. Each subnetwork consists of an independent linear transformation layer and an output layer. Each output layer outputs the predicted probability distribution through the sigmoid function.
9. A safety management and control system for highway concrete guardrail construction sites according to claim 1, characterized in that, Based on the semantic accountability results, a security control instruction vector is generated as follows: For each candidate target, the behavioral fingerprint deviation is calculated based on the behavioral fingerprint vector of the current monitoring period and the acquired historical normal construction process behavioral fingerprint vector. Obtain the number of abnormal windows for candidate targets, and calculate the average abnormal score for the current monitoring period based on the abnormal score sequence; A comprehensive risk index for candidate targets is obtained based on behavioral fingerprint deviation, number of abnormal windows, and average abnormal score during the current monitoring period. The semantic accountability results corresponding to each abnormal window are aggregated to obtain the set of semantic accountability results for the candidate target in the current monitoring period, and a safety behavior profile record vector is constructed for each candidate target based on the set of semantic accountability results. Risk level identifiers are determined based on comprehensive risk indicators, including low risk level, medium risk level, and high risk level. Based on the risk level identifier and the semantic judgment result set, a safety control instruction vector corresponding to the candidate target is generated.