Regional target and environment dynamic sensing method for construction site video monitoring

Abstract

The invention discloses a regional target and environment dynamic sensing method for construction site video monitoring, and the method comprises the steps: carrying out the initial deployment of distributed cameras for a construction site, and carrying out the deployment optimization; collecting video data based on Kafka Broker cluster multithreading, collecting environmental parameters, and performing fusion based on Mahalanobis distance to obtain fused environmental parameters; performing multi-target detection on the collected video data, and analyzing behavior characteristics of the collected video data to obtain video target data; environment parameter feature representation is extracted from the fused environment parameters, multi-modal dynamic fusion is carried out on the environment parameter feature representation and video target data based on learnable parameters, and a comprehensive situation is analyzed through a situation awareness model to obtain real-time comprehensive situation data; and performing real-time risk assessment on personnel behaviors and environmental conditions in the construction site by adopting Bayesian network analysis based on the real-time comprehensive situation data, identifying risk events, and automatically generating alarm information. According to the invention, the intelligent level of construction site video monitoring is obviously improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring technology, and in particular to a method for dynamic perception of regional targets and environments for construction site video monitoring. Background Art

[0002] With the continuous advancement of urbanization and the rapid development of infrastructure construction, the number and scale of construction sites are increasing year by year. As a dynamic and high-risk working environment, construction sites face many safety hazards and management difficulties due to frequent personnel turnover, complex work tasks, and changing environmental factors. In order to improve the safety and management efficiency of construction site operations, traditional video surveillance methods have been widely used, but due to their technical limitations, they have not yet met the actual needs of modern intelligent management of construction sites.

[0003] Existing methods for video surveillance of construction sites mainly rely on fixed-angle cameras to cover the area and analyze the surveillance images manually or through simple algorithms. However, these methods have the following shortcomings: (1) Insufficient optimization deployment of monitoring areas Traditional camera deployment methods are usually based on experience or simple geometric analysis, which makes it difficult to optimize complex construction site scenarios. For example, there may be blind spots in the monitoring area, or key areas may not be effectively covered due to building obstructions, thus reducing the comprehensiveness and reliability of the monitoring system.

[0004] (2) Poor fusion capability of multimodal data The construction site environment includes not only human activities, but also many environmental parameters (such as temperature and humidity, gas concentration, light intensity, etc.), which are crucial for situational awareness and risk assessment. However, traditional monitoring methods lack the ability to integrate and analyze multiple types of data in real time, and cannot achieve dynamic association between video data and environmental parameters, thus limiting the comprehensive perception of the overall situation.

[0005] (3) Target detection and behavior analysis are not accurate enough In construction site video surveillance, accurate detection and identification of multiple targets (such as people) and their behavioral characteristics are the basis of situational awareness. However, due to the complex background of construction site scenes, severe occlusion of multiple targets, and rapid behavioral changes, traditional target detection algorithms have significant deficiencies in accuracy and robustness, and are difficult to meet actual needs.

[0006] (4) Lack of dynamic risk assessment capabilities The main purpose of construction site monitoring systems is to identify and warn of potential safety risks. However, existing methods often rely on a single data source or simple rules when evaluating dynamic risk events, which cannot achieve a comprehensive analysis of human behavior and environmental conditions, and are prone to missed reports or false reports.

[0007] (5) Inefficient use of computing resources In large-scale construction site monitoring scenarios, the amount of video data is huge and the sensor information is complex. Traditional systems have problems such as unreasonable allocation of computing resources and low multi-threaded concurrency efficiency during the processing process, which limits real-time performance and response speed.

[0008] In view of the above problems, an intelligent method for construction site video surveillance is urgently needed. Summary of the invention

[0009] In order to overcome the shortcomings of the prior art, the present invention provides a method for dynamic perception of regional targets and environments for construction site video surveillance, which is used to solve the shortcomings of existing construction site video surveillance methods in monitoring coverage, data fusion accuracy, target detection accuracy, risk assessment efficiency and resource utilization efficiency, thereby significantly improving the intelligence level of construction site video surveillance.

[0010] To solve the above problems, the technical solution adopted by the present invention is as follows: A method for dynamic perception of regional targets and environments for construction site video monitoring includes the following steps: Initially deploy distributed cameras on the construction site and optimize the deployment by discretizing units and obtaining the distance and angle between units; By deploying optimized distributed cameras to collect video data based on the multi-threaded Kafka Broker cluster, environmental parameters are collected through multiple types of environmental sensors and fused based on the Mahalanobis distance to obtain fused environmental parameters. Perform multi-target detection on the collected video data and analyze its behavior characteristics to obtain video target data; Based on the sensor neural network, the environmental parameter feature representation is extracted from the fused environmental parameters, and multi-modal dynamic fusion is performed with the video target data based on learnable parameters. The comprehensive situation is analyzed through the RNN construction site situation awareness model to obtain real-time comprehensive situation data. Based on real-time comprehensive situation data, Bayesian network analysis is used to conduct real-time risk assessment of personnel behavior and environmental conditions on the construction site, identify risk events, and automatically generate alarm information; Among them, when performing multi-target detection, it includes: obtaining the target position of the key frame by building a construction site target detector; capturing spatial semantic information and dynamic information through different paths by building a construction site feature extraction model to generate a fusion feature map, and completing behavior classification in combination with the target position of the key frame.

[0011] As a preferred implementation mode of the present invention, when performing deployment optimization, it includes: Discretize the construction site into units to obtain several site units; Get the site unit where the distributed camera is set , and get the site unit corresponding to the direction of the distributed camera ; Acquire field units from critical areas that must be video monitored , and obtain the site unit according to the distance between the site units , site unit The angle between If the angle is less than the preset value, it is considered that the distributed camera can detect the site unit. Conduct monitoring coverage; By obtaining the site unit corresponding to each site unit and the direction of the distributed camera The angle between them is used to determine the monitoring coverage area of ​​the distributed cameras.

[0012] As a preferred embodiment of the present invention, after determining the monitoring coverage area of ​​the distributed camera, it includes: Determine the first edge site unit of the building that blocks monitoring and the second edge site unit ; Get the first edge site unit , Second edge site unit and venue units The angle formed ; Identify a site unit within the monitoring coverage area , according to its relationship with the first edge site unit , Second edge site unit , site unit The angle between the two, the distance between the site units and the angle , determine the judgment parameters The value of Whether it is blocked by buildings; According to the monitoring coverage area and judgment parameters , determine the site units in the key area that cannot be covered by monitoring or are blocked by buildings, and then adjust the distributed cameras so that all site units in the key area can be covered by monitoring and are not blocked by buildings.

[0013] As a preferred embodiment of the present invention, when collecting video data, it includes: Receive video data, parse and encapsulate it, and then send the video message data to the Kafka message queue for storage after serialization; Create multiple receivers with the same groupId for the Kafka data in the same Topic, pull the Kafka data from different Partitions based on the thread pool, store them in the form of data blocks, and aggregate the Kafka data pulled by multiple receivers; After deserializing the aggregated Kafka data, the obtained Json message is converted into Json data format, and the Json data is parsed and converted into Mat data type that can be used for image processing.

[0014] As a preferred embodiment of the present invention, when sending to the Kafka message queue for storage, it includes: Send the video message data to the corresponding Topic according to the distributed camera ID, and store the video message data in the corresponding Partition according to the acquisition angle corresponding to the video message data; Each Topic contains multiple Partitions. A Kafka Broker cluster node is configured for each Partition in each Topic to form a Kafka Broker cluster. The number of threads in the thread pool is consistent with the number of Partitions of the corresponding Topic.

[0015] As a preferred embodiment of the present invention, when fusion is performed based on Mahalanobis distance, it includes: Obtain data detected by different types of environmental sensors and The Mahalanobis distance of Using the Mahalanobis distance as the trust function, we get the data and Between function; Using the Function, generate data The credibility of the data is obtained by using the credibility The information entropy of and The Mahalanobis distance of Data and Data The difference in the average Mahalanobis distance of The new credibility is normalized to obtain the data The basic probability distribution value of ; Use the basic probability distribution value to eliminate abnormal data and The basic probability distribution values ​​are subjected to data fusion through combination rules to obtain the fusion environment parameters.

[0016] As a preferred embodiment of the present invention, when obtaining the target position of the key frame, it includes: Divide the video data into multiple time periods and extract key frames from each time period; Build improvements As a construction site target detector, the target position is detected by inputting key frames into the construction site target detector and inputting Modules; When capturing spatial semantic information and dynamic information in different paths to generate fused feature maps, it includes: Build improvements The model is used as a site feature extraction model and inputs the collected video data; pass The path operates at low temporal resolution and high spatial resolution, capturing the spatial semantic information in the video data and outputting the first feature map; pass The path operates at high temporal resolution and low spatial resolution, capturing dynamic information of fast-changing behaviors in video data, and adopts a smaller convolution width to output the second feature map; The second feature map is scaled and fused with the first feature map to obtain a fused feature map, and the fused feature map is input into Modules; When completing the behavior classification in combination with the target position of the keyframe, it includes: pass The module projects the target frame generated based on the target position information onto the fused feature map to obtain the corresponding feature matrix, and generates a fused feature map of uniform size. After pooling, full connection and behavior classification, the video target data is obtained. Among them, improvement In the detection head Module, in the neck network Module replacement for lightweight module.

[0017] As a preferred embodiment of the present invention, in constructing an improved Models include: Introducing the attention module for the time dimension into Path and In the residual module of the path, the contribution of weights is used to improve the attention mechanism; Among them, when using the contribution of weights to improve the attention mechanism, it includes: obtaining the variance and weight of each time dimension of the video data processed each time; After temporal attention extraction based on the weight of the time dimension and the variance of the time dimension, the first feature map and the second feature map are output, as shown in Formula 13: (13); In the formula, is the first characteristic map or the second characteristic map, is the activation function, is the combination of weight and variance for each time dimension, is the variance of each time dimension, is the weight of each time dimension, The video data processed each time.

[0018] As a preferred embodiment of the present invention, when extracting the characteristic representation of environmental parameters, it includes: Construct a sensor neural network based on three-dimensional convolution and input the pre-processed fusion environment parameters; After the first After the convolutional layer processing, it passes through Activation function and The maximum pooling layer, and then the second Convolutional layer, third The convolutional layer processes the feature, reducing the spatial and temporal resolution of the feature, normalizing it through the batch normalization layer, and The convolutional layer outputs the characteristic representation of environmental parameters; Among them, through the second Convolutional layer, third The convolutional layer uses multiple dilation rates Cavity convolution extracts features of large-scale targets; In the feature extraction process, a time series attention module is added to adaptively adjust the feature weight of each fused environmental parameter and highlight the feature representation of the environmental parameter. Residual connection and feature pyramid structure are used to integrate features at different levels and provide multi-scale feature representation.

[0019] As a preferred embodiment of the present invention, when analyzing the comprehensive situation through the RNN construction site situation awareness model, it includes: For each site unit in the construction site, combined with the personnel behavior rules and environmental parameter standards corresponding to each site unit, various anticipated behavioral risk events and environmental risk events are scanned one by one to form a behavior-environmental risk event rule dictionary; Through the RNN network model, the behavior-environment risk event rule dictionary and the behavior-environment routine event rule dictionary are deeply learned, and the nonlinear mapping relationship between the video-environment fusion features and the behavior risk events and environmental risk events is established to obtain the RNN construction site situation awareness model; The real-time video-environment fusion features are used The feature error correction model performs cross-identification to obtain an identification feature set; the identification feature set is input into the RNN construction site situation awareness model for identification to obtain real-time comprehensive situation data; Among them, when conducting deep learning, it includes: adding The algorithm randomly discards some neurons by introducing The algorithm updates the parameters using an exponential decay algorithm for the learning rate.

[0020] Compared with the prior art, the present invention has the following beneficial effects: (1) Improve the coverage and deployment rationality of construction site monitoring By dividing the construction site into discrete units and optimizing the deployment of cameras based on the distance and angle relationship between site units, distributed cameras can effectively cover key areas and reduce blind spots.

[0021] (2) Realizing dynamic fusion of environmental multimodal data A multi-sensor data fusion algorithm based on Mahalanobis distance is used to eliminate abnormal environmental data, ensure data credibility, and generate high-precision fusion environmental parameters through combination rules.

[0022] In the sensor neural network, deep feature extraction is performed on the fused environmental parameters, and comprehensive and accurate environmental dynamic perception capabilities are provided through time series attention modules and multi-scale feature integration.

[0023] (3) Improving the accuracy and robustness of target detection and behavior analysis An improved construction site target detector and construction site feature extraction model are used to capture the spatial semantic information and behavioral dynamic information of construction site targets through the fusion of high and low temporal resolution and spatial resolution of multiple paths.

[0024] The introduction of lightweight model design improves computational efficiency while maintaining detection accuracy, which is particularly suitable for multi-target detection and behavior analysis in complex construction site environments.

[0025] (4) Accurate analysis of the overall situation and dynamic risk assessment Based on the RNN construction site situation awareness model, the video target data and environmental parameter characteristics are integrated to establish a nonlinear mapping relationship between behavioral risk events and environmental risk events.

[0026] Through the construction of a behavior-environment risk event rule dictionary and deep learning optimization, it is possible to achieve comprehensive situation analysis of various complex construction site risk scenarios and identify potential risk events.

[0027] (5) Automated real-time risk warning and alarm information generation Through real-time comprehensive situation data and Bayesian network analysis, the behavior of personnel and environmental conditions on the construction site are evaluated in real time to identify medium and high-risk events.

[0028] (6) Efficiency and stability of data collection and processing The multi-threaded design based on the Kafka Broker cluster enables efficient collection, storage and processing of massive video data and environmental parameters, ensuring the real-time and stability of the system.

[0029] Through thread pool and distributed storage strategy, the performance bottleneck problem of traditional monitoring system in large-scale data processing is solved.

[0030] (7) Supporting the application and expansion of complex construction site scenarios Through the three-dimensional convolution structure and time series attention module of the sensor neural network, it can adapt to a variety of complex construction site scenarios (such as large-scale construction sites and dynamic construction environments).

[0031] The system design is highly modular and can be easily expanded according to actual application needs, such as adding new sensor types or connecting more video data sources.

[0032] (8) Algorithm optimization and improvement of computing resource utilization efficiency Introducing attention mechanism, lightweight module and dilated convolution in target detection and feature extraction can greatly reduce computational overhead and improve processing efficiency.

[0033] The random neuron dropout algorithm and learning rate exponential decay mechanism are added to the deep learning model to significantly improve the generalization ability and stability of the model.

[0034] In summary, the present invention can significantly improve the intelligence level of construction site video surveillance, achieve a comprehensive improvement in monitoring coverage, data fusion accuracy, target detection accuracy and risk assessment efficiency, provide reliable technical support for construction site safety management, and has important application value and practical significance.

[0035] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a step diagram of the method for dynamic perception of regional targets and environment for construction site video monitoring provided by the present invention; Figure 2 It is a real-time risk assessment flow chart provided by the present invention. DETAILED DESCRIPTION

[0037] The method for dynamic perception of regional targets and environment for construction site video monitoring provided by the present invention is as follows: Figure 1As shown, the following steps are included: Step S1: Initially deploy distributed cameras on the construction site and optimize the deployment by discretizing units and obtaining the distance and angle between units; Step S2: Deploy optimized distributed cameras to collect video data based on the Kafka Broker cluster multi-threading, and collect environmental parameters through multiple types of environmental sensors and fuse them based on Mahalanobis distance to obtain fused environmental parameters; Step S3: Perform multi-target detection on the collected video data through a construction site target detector combined with a construction site feature extraction model, and analyze its behavior characteristics to obtain video target data; Step S4: extracting environmental parameter feature representation from fused environmental parameters based on sensor neural network, and performing multi-modal dynamic fusion with video target data based on learnable parameters, analyzing the comprehensive situation through the RNN construction site situation awareness model, and obtaining real-time comprehensive situation data; Step S5: Based on the real-time comprehensive situation data, a real-time risk assessment is conducted on the personnel behavior and environmental conditions in the construction site, medium-risk events and high-risk events are identified, alarm information is automatically generated, and related control systems are linked to execute emergency response.

[0038] Specifically, the multiple types of environmental sensors include: temperature sensors, humidity sensors, light intensity sensors, gas sensors, noise sensors, and particle and dust sensors.

[0039] Environmental parameters include: temperature, humidity, light intensity, gas concentration and noise.

[0040] In the above step S1, when the deployment optimization is performed by unit discretization segmentation and acquisition of the distance and angle between units, it includes: Discretize the construction site into units to obtain several site units; Get the site unit where the distributed camera is set , and get the site unit corresponding to the direction of the distributed camera ; Acquire field units from critical areas that must be video monitored , and obtain the site unit according to the distance between the site units , site unit The angle between them is shown in Formula 1: (1); In the formula, is the angle, For site unit and site units The distance between For site unit and site units The distance between For site unit and site units The distance between For site unit and site units The distance between If the angle is less than the preset value, it is considered that the distributed camera can detect the site unit. Conduct monitoring coverage; By obtaining the site unit corresponding to each site unit and the direction of the distributed camera The angle between them is used to determine the monitoring coverage area of ​​the distributed cameras.

[0041] Furthermore, after determining the monitoring coverage area of ​​the distributed cameras, it includes: Determine the first edge site unit of the building that blocks monitoring and the second edge site unit ; Get the first edge site unit , Second edge site unit and venue units The angle formed ; Identify a site unit within the monitoring coverage area , and get the first edge site unit , site unit and venue units The angle formed ; Get the second edge site unit , site unit and venue units The angle formed ; Get Angle The maximum angle of the field unit is determined based on the maximum angle. Whether it is blocked by buildings, as shown in Formula 2: (2); In the formula, To determine the parameters, Angle The maximum angle of To obtain , The larger value of For site unit To site unit The distance between First edge site unit To site unit The distance between Second edge site unit To site unit distance; If the judgment parameter If it is 1, it means the site unit It is within the surveillance coverage area but blocked by buildings; According to the monitoring coverage area and judgment parameters , determine the site units in the key area that cannot be monitored by distributed cameras or are blocked by buildings; By adjusting the setting position of distributed cameras and / or adding distributed cameras, all site units in the key area can be monitored and covered by distributed cameras and are not blocked by buildings.

[0042] In the above step S2, when collecting video data based on the Kafka Broker cluster multi-threading, it includes: Receive video data from distributed cameras, parse and encapsulate it, and then serialize and send the video message data to the Kafka message queue for storage; Create a KafkaInputDStream object as a subclass of ReceiverInputDStream, and use the KafkaInputDStream object to create multiple KafkaReceiver receivers with the same groupId for the Kafka data in the same Topic to pull Kafka data from different Partition partitions, and use the union() method to aggregate the Kafka data pulled by multiple KafkaReceiver receivers; Deserialize the aggregated Kafka data and convert the acquired Json message into Json data format. Use the OpenCV vision library to parse and convert the Json data into Mat data type that can be used for image processing. Among them, when sending to the Kafka message queue for storage, it includes: By obtaining the Key value of the video message data, determine the distributed camera ID to which the video message data belongs, and send the video message data to the corresponding Topic according to the distributed camera ID, and store the video message data in the corresponding Partition partition according to the acquisition angle corresponding to the video message data; Each Topic contains multiple Partitions. By creating a Kafka Producer object, a Kafka Broker cluster node is configured for each Partition in each Topic to form a Kafka Broker cluster.

[0043] Furthermore, when pulling Kafka data from different Partitions, it includes: Call the onStart() method to start receiving Kafka data, establish a connection with the Kafka Broker cluster, and obtain the consumerConnector connection object; Use the consumerConnector connection object to create a TopicMessageStreams table List[KafkaStream] that stores the mapping relationship between the Topic and its Partition partition; Create a thread pool executorPool for each Topic, and consume messages through the thread pool executorPool; Create a MessageHandler message processor for each Partition partition data stream, and configure a processor thread for the MessageHandler message processor in the thread pool executorPool; After the processor thread is started, the KafkaReceiver receiver will pull the Kafka data and pass it to the ReceiverSupervisor for processing; ReceiverSupervisor stores the received Kafka data in the form of data blocks through BlockManager; ReceiverSupervisor sends a message encapsulating a data block as a parameter to ReceiverTracker, which is the scheduler, through the AddBlock method to notify it of the newly added data block; ReceiverTracker assigns the received data blocks to batches at preset intervals as input data for the elastic distributed dataset; Among them, each element in the mapping relationship table List[KafkaStream] represents a partition data stream of this Topic, and the number of threads in the thread pool executorPool is consistent with the number of partitions of its corresponding Topic.

[0044] In the above step S2, when the fusion is performed based on the Mahalanobis distance, it includes: Obtain data detected by different types of environmental sensors and The Mahalanobis distance is shown in Formula 3: (3); In the formula, For data and The Mahalanobis distance, For transposition, is the covariance matrix of the multidimensional random variable; Using Mahalanobis distance as the trust function, we get the data and Between Function, as shown in Formula 4: (4); In the formula, For data and Between function; use Function, generate data credibility, and use the credibility to obtain data The information entropy of The new credibility is shown in Formula 5, Formula 6, and Formula 7: (5); In the formula, For data credibility, For data and data The number of (6); In the formula, For data Information entropy of (7); In the formula, For data New credibility, It’s data and The Mahalanobis distance of Data and Data The difference in the average Mahalanobis distance is shown in Formula 8: (8); In the formula, For data and The Mahalanobis distance, for Data and Data The average Mahalanobis distance of Order Data The new credibility is normalized to obtain data The basic probability distribution value of is shown in Formula 9: (9); In the formula, For data The basic probability distribution value of ; Use basic probability to assign values ​​to data Remove abnormal data and use the remaining data The basic probability distribution value of The combination rules of evidence theory are used to perform data fusion and obtain fusion environment parameters.

[0045] In the above step S3, when performing multi-target detection on the collected video data, it includes: The collected video data is divided into multiple time periods according to the preset time intervals, and the preset sampling frame number is used. and step length Extract key frames from each time period, as shown in formula 10: (10); In the formula, is a key frame; Build improvements As a construction site target detector, the target position is detected by inputting the key frame into the construction site target detector, and the detected target position information is input into Modules; Build improvements The model is used as a construction site feature extraction model. The collected video data is input into the construction site feature extraction model. The path operates at low temporal resolution and high spatial resolution, capturing the spatial semantic information in the video data and outputting the first feature map; Through the site feature extraction model The path runs at high temporal resolution and low spatial resolution to capture the dynamic information of fast-changing behaviors in video data, and uses a smaller convolution width to reduce the channel capacity to reduce the amount of calculation and output the second feature map; The second feature map is passed through After the convolution is scaled, it is fused with the first feature map to obtain a fused feature map, and the fused feature map is input into Modules; pass The module projects the target frame generated based on the target position information onto the fusion feature map to obtain the corresponding feature matrix. Expanded by time dimension , generate a fusion feature map of uniform size, and after processing by the pooling layer and the fully connected layer, use The classifier performs behavior classification to obtain video target data; Among them, improvement In the detection head Module, in the neck network Module replacement for lightweight Modules; pass The module improves the model's utilization of multi-scale features by making it lightweight The module improves the neck network structure, reduces the model parameters and calculation amount, and shortens the model reasoning time; Path and The paths all include: a convolutional layer, a pooling layer, a first residual module, a second residual module, a third residual module, and a fourth residual module connected in sequence; improve The length of video data processed by the model each time is Frame, where Path in steps sampling Frames are processed, Path in steps sampling Frames to process.

[0046] Further, in building improvements When the model is used as a site feature extraction model, it includes: Introducing the attention module for the time dimension into Path and In the residual module of the path, the contribution of weights is used to improve the attention mechanism; Among them, when using the contribution of weights to improve the attention mechanism, it includes: obtaining the variance of each time dimension of the video data processed each time, as shown in Formula 11: (11); In the formula, is the variance of each time dimension, For each processed video data, is the mean value of the video data processed each time, is the standard deviation of the video data processed each time, and are the trainable radiation transformation parameters for scale and displacement, respectively. is a constant used to prevent the variance from being zero, is the attention module for the time dimension; Get the weight of each time dimension, as shown in Formula 12: (12); In the formula, is the weight of each time dimension, is the scaling factor for each time dimension, used to represent the intensity of change over time; is the number of time dimensions; After temporal attention extraction based on the weight of the time dimension and the variance of the time dimension, the first feature map and the second feature map are output, as shown in Formula 13: (13); In the formula, is the first characteristic map or the second characteristic map, is the activation function, is the combination of the weight and variance for each time dimension.

[0047] In the above step S4, when extracting the characteristic representation of the environmental parameters, it includes: Construct a sensor neural network based on three-dimensional convolution, the sensor neural network includes: first Convolutional layer, Max pooling layer, second Convolutional layer, third Convolutional layer, batch normalization layer, and the fourth Convolutional layers; After preprocessing, the fusion environment parameters are input into the sensor neural network. 32 of the convolutional layers After convolution kernel processing, it passes through Activation function and The maximum pooling layer, and then the second 64 of the convolutional layers Convolution kernel, third 128 in the convolutional layer Convolution kernel processing, while reducing the spatial and temporal resolution of features, normalized through batch normalization layer, through the fourth Convolutional layer usage The convolution kernel outputs 256 channels of environmental parameter feature representation; Among them, by using multiple expansion rates in the middle layer of the sensor neural network Cavity convolution extracts features of large-scale targets; In the feature extraction process, a time series attention module is added, and the correlation between adjacent fusion environment parameters is obtained through the time series attention module. The feature weight of each fusion environment parameter is adaptively adjusted and the feature representation of the environment parameter is highlighted. The residual connection and feature pyramid structure are used to alleviate the gradient disappearance of the deep network. At the same time, the feature pyramid structure is used to integrate features at different levels to provide multi-scale feature representation.

[0048] In the above step S4, when performing multi-modal dynamic fusion, it includes: Through learnable parameters The weights of the video target data and the environmental parameter feature representation are dynamically adjusted, and multimodal fusion is performed according to the dynamically adjusted weights to obtain the video-environment fusion feature, as shown in Formula 13: (13); In the formula, is the video-environment fusion feature, is the characteristic representation of environmental parameters, is the video target data; Among them, the learnable parameters , whose value is dynamically generated according to the statistics of the input features to adaptively adjust the fusion strategy according to different construction site scenarios.

[0049] In the above step S4, when analyzing the comprehensive situation through the RNN construction site situation awareness model, it includes: For each site unit in the construction site, combined with the personnel behavior rules and environmental parameter standards corresponding to each site unit, various anticipated behavioral risk events and environmental risk events are scanned one by one to form a behavior-environmental risk event rule dictionary; Provide a behavior-environment routine event rule dictionary, conduct deep learning on the behavior-environment risk event rule dictionary and the behavior-environment routine event rule dictionary through the RNN network model, establish a nonlinear mapping relationship between video-environment fusion features and behavior risk events and environmental risk events, and obtain the RNN construction site situation awareness model, as shown in Formula 14: (14); In the formula, is the video-environment fusion feature, is the number of video-environment fusion features, Behavioral risk events, is the number of behavioral risk events, For environmental risk events, is the number of environmental risk events, is the nonlinear mapping relationship between video-environment fusion features and behavioral risk events. It is the nonlinear mapping relationship between video-environment fusion features and environmental risk events; The real-time video-environment fusion features are used The feature error correction model performs cross-identification to obtain a set of identification features; The identification feature set is input into the RNN construction site situation awareness model for identification to obtain real-time comprehensive situation data; Among them, when conducting deep learning, it includes: The video-environment fusion features, various behavioral risk events, and various environmental risk events are coded in a numbered manner. Normalization: Normalize the data and add The algorithm changes the structure of the neural network by randomly discarding some neurons, reducing the dependencies between neurons, and introducing The algorithm and the exponential decay algorithm of the learning rate update parameters to speed up the convergence of the model.

[0050] In the above step S5, if Figure 2 As shown, when conducting a real-time risk assessment, include: The real-time comprehensive situation data is analyzed for correlation using Bayesian network analysis and behavior-environment risk event rule dictionary to determine whether there are strongly correlated behavior events and / or environmental events, or whether there are moderately correlated behavior events and / or environmental events; If they exist, strongly associated behavioral events are identified as high-risk behavioral events, and strongly associated environmental events are identified as high-risk environmental events; Moderately correlated behavioral events are identified as medium-risk behavioral events, and moderately correlated environmental events are identified as medium-risk environmental events.

[0051] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. A method for dynamic perception of regional targets and environment for construction site video monitoring, characterized in that: The following steps are involved: Initially deploy distributed cameras on the construction site and optimize the deployment by discretizing units and obtaining the distance and angle between units; By deploying optimized distributed cameras to collect video data based on the multi-threaded Kafka Broker cluster, environmental parameters are collected through multiple types of environmental sensors and fused based on the Mahalanobis distance to obtain fused environmental parameters. Perform multi-target detection on the collected video data and analyze its behavior characteristics to obtain video target data; Based on the sensor neural network, the environmental parameter feature representation is extracted from the fused environmental parameters, and multi-modal dynamic fusion is performed with the video target data based on learnable parameters. The comprehensive situation is analyzed through the RNN construction site situation awareness model to obtain real-time comprehensive situation data. Based on real-time comprehensive situation data, Bayesian network analysis is used to conduct real-time risk assessment of personnel behavior and environmental conditions on the construction site, identify risk events, and automatically generate alarm information; Among them, when performing multi-target detection, it includes: obtaining the target position of the key frame by building a construction site target detector; capturing spatial semantic information and dynamic information through different paths by building a construction site feature extraction model to generate a fusion feature map, and completing behavior classification in combination with the target position of the key frame.

2. The method for dynamic perception of regional targets and environment for construction site video monitoring according to claim 1, characterized in that: When optimizing your deployment, include: Discretize the construction site into units to obtain several site units; Get the site unit where the distributed camera is set , and get the site unit corresponding to the direction of the distributed camera ; Acquire field units from critical areas that must be video monitored , and obtain the site unit according to the distance between the site units , site unit The angle between If the angle is less than the preset value, it is considered that the distributed camera can detect the site unit. Conduct monitoring coverage; By obtaining the site unit corresponding to each site unit and the direction of the distributed camera The angle between them is used to determine the monitoring coverage area of ​​the distributed cameras.

3. The method for dynamic perception of regional targets and environment for construction site video monitoring according to claim 2 is characterized in that: After determining the surveillance coverage area of ​​the distributed cameras, including: Determine the first edge site unit of the building that blocks monitoring and the second edge site unit ; Get the first edge site unit , Second edge site unit and venue units The angle formed ; Identify a site unit within the monitoring coverage area , according to its relationship with the first edge site unit , Second edge site unit , site unit The angle between the two, the distance between the site units and the angle , determine the judgment parameters The value of Whether it is blocked by buildings; According to the monitoring coverage area and judgment parameters , determine the site units in the key area that cannot be covered by monitoring or are blocked by buildings, and then adjust the distributed cameras so that all site units in the key area can be covered by monitoring and are not blocked by buildings.

4. The method for dynamic perception of regional targets and environments for construction site video monitoring according to claim 3 is characterized in that: When collecting video data, include: Receive video data, parse and encapsulate it, and then send the video message data to the Kafka message queue for storage after serialization; Create multiple receivers with the same groupId for the Kafka data in the same Topic, pull the Kafka data from different Partitions based on the thread pool, store them in the form of data blocks, and aggregate the Kafka data pulled by multiple receivers; After deserializing the aggregated Kafka data, the obtained Json message is converted into Json data format, and the Json data is parsed and converted into Mat data type that can be used for image processing.

5. The method for dynamic perception of regional targets and environment for construction site video monitoring according to claim 4 is characterized in that: When sending to the Kafka message queue for storage, it includes: Send the video message data to the corresponding Topic according to the distributed camera ID, and store the video message data in the corresponding Partition according to the acquisition angle corresponding to the video message data; Each Topic contains multiple Partitions. A Kafka Broker cluster node is configured for each Partition in each Topic to form a Kafka Broker cluster. The number of threads in the thread pool is consistent with the number of Partitions of the corresponding Topic.

6. The method for dynamic perception of regional targets and environments for construction site video monitoring according to any one of claims 1 to 5, characterized in that: When fusion is performed based on Mahalanobis distance, it includes: Obtain data detected by different types of environmental sensors and The Mahalanobis distance of Using the Mahalanobis distance as the trust function, we get the data and Between function; Using the Function, generate data The credibility of the data is obtained by using the credibility The information entropy of and The Mahalanobis distance of Data and Data The difference in the average Mahalanobis distance of The new credibility is normalized to obtain the data The basic probability distribution value of ; Use the basic probability distribution value to eliminate abnormal data and The basic probability distribution values ​​are subjected to data fusion through combination rules to obtain the fusion environment parameters.

7. The method for dynamic perception of regional targets and environment for construction site video monitoring according to claim 6, characterized in that: When obtaining the target position of a keyframe, include: Divide the video data into multiple time periods and extract key frames from each time period; Build improvements As a construction site target detector, the target position is detected by inputting key frames into the construction site target detector and inputting Modules; When capturing spatial semantic information and dynamic information in different paths to generate fused feature maps, it includes: Build improvements The model is used as a site feature extraction model and inputs the collected video data; pass The path operates at low temporal resolution and high spatial resolution, capturing the spatial semantic information in the video data and outputting the first feature map; pass The path operates at high temporal resolution and low spatial resolution, capturing dynamic information of fast-changing behaviors in video data, and adopts a smaller convolution width to output the second feature map; The second feature map is scaled and fused with the first feature map to obtain a fused feature map, and the fused feature map is input into Modules; When completing the behavior classification in combination with the target position of the keyframe, it includes: pass The module projects the target frame generated based on the target position information onto the fused feature map to obtain the corresponding feature matrix, and generates a fused feature map of uniform size. After pooling, full connection and behavior classification, the video target data is obtained. Among them, improvement In the detection head Module, in the neck network Module replacement for lightweight module.

8. The method for dynamic perception of regional targets and environment for construction site video monitoring according to claim 7, characterized in that: Improvements in building Models include: Introducing the attention module for the time dimension into Path and In the residual module of the path, the contribution of weights is used to improve the attention mechanism; Among them, when using the contribution of weights to improve the attention mechanism, it includes: obtaining the variance and weight of each time dimension of the video data processed each time; After temporal attention extraction based on the weight of the time dimension and the variance of the time dimension, the first feature map and the second feature map are output, as shown in Formula 13: （13）； In the formula, is the first characteristic map or the second characteristic map, is the activation function, is the combination of weight and variance for each time dimension, is the variance of each time dimension, is the weight of each time dimension, The video data processed each time.

9. The method for dynamic perception of regional targets and environment for construction site video monitoring according to claim 8, characterized in that: When extracting the characteristic representation of environmental parameters, it includes: Construct a sensor neural network based on three-dimensional convolution and input the pre-processed fusion environment parameters; After the first After the convolutional layer processing, it passes through Activation function and The maximum pooling layer, and then the second Convolutional layer, third The convolutional layer processes the feature, reducing the spatial and temporal resolution of the feature, normalizing it through the batch normalization layer, and The convolutional layer outputs the characteristic representation of environmental parameters; Among them, through the second Convolutional layer, third The convolutional layer uses multiple dilation rates Cavity convolution extracts features of large-scale targets; In the feature extraction process, a time series attention module is added to adaptively adjust the feature weight of each fused environmental parameter and highlight the feature representation of the environmental parameter. Residual connection and feature pyramid structure are used to integrate features at different levels and provide multi-scale feature representation.

10. The method for dynamic perception of regional targets and environments for construction site video monitoring according to claim 9, characterized in that: When analyzing the comprehensive situation through the RNN construction site situation awareness model, it includes: For each site unit in the construction site, combined with the personnel behavior rules and environmental parameter standards corresponding to each site unit, various anticipated behavioral risk events and environmental risk events are scanned one by one to form a behavior-environmental risk event rule dictionary; Through the RNN network model, the behavior-environment risk event rule dictionary and the behavior-environment routine event rule dictionary are deeply learned, and the nonlinear mapping relationship between the video-environment fusion features and the behavior risk events and environmental risk events is established to obtain the RNN construction site situation awareness model; The real-time video-environment fusion features are used The feature error correction model performs cross-identification to obtain an identification feature set; the identification feature set is input into the RNN construction site situation awareness model for identification to obtain real-time comprehensive situation data; Among them, when conducting deep learning, it includes: adding The algorithm randomly discards some neurons by introducing The algorithm updates the parameters using an exponential decay algorithm for the learning rate.

Images