Unmanned aerial vehicle construction site safety risk intelligent monitoring system and method integrated with deep learning

Abstract

The invention discloses an unmanned aerial vehicle construction site safety risk intelligent monitoring system integrated with deep learning. The unmanned aerial vehicle construction site safety risk intelligent monitoring system comprises a GPS positioning module, an air monitoring module, a danger identification operation module, a voice alarm module and a self-service charging fixed patrol module. The GPS positioning module generates three-dimensional coordinates and positions the unmanned aerial vehicle; the air monitoring module monitors the particle concentration of a set substance, and transmits a signal to the voice alarm module when the particle concentration exceeds a set value; when the value is not higher than the set value, signal transmission is stopped; the danger identification operation module shoots a construction site image and judges whether a dangerous behavior exists or not; if the dangerous behavior exists, the signal is transmitted to the voice alarm module, the dangerous behavior disappears, and signal transmission is stopped; after receiving the signal, the voice alarm module gives out a voice alarm, obtains the three-dimensional coordinates of the hazard source through the GPS positioning module, and transmits the coordinates and the signal to a supervisor; the invention further discloses a corresponding monitoring method. Construction safety is guaranteed, operation cost is reduced, and accuracy of unmanned aerial vehicle monitoring is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of construction site safety monitoring, and more specifically, to an intelligent monitoring system and method for construction site safety risks of unmanned aerial vehicles integrated with deep learning. Background Art

[0002] As an achievement of the cross-integration of multiple disciplines, unmanned aerial vehicle (UAV) technology originated in the early 20th century. Starting from the initial flying bombs and target drones, it has developed to be widely used in multiple fields today. The market scale is constantly expanding, and there are both technological innovations and challenges. China's policy support and complete industrial chain also contribute to its development. However, in the field of construction, there are deficiencies in the traditional safety management applications based on UAVs. Although UAVs can collect certain data, it is difficult to accurately and real-time monitor the positions of personnel. In a complex construction site environment, it is impossible to accurately judge whether personnel have entered a dangerous area. The equipment and systems carried lack an intelligent early warning mechanism, and the endurance and flight stability of UAVs themselves are limited, resulting in limited ability to prevent construction site accidents and making it difficult to fully play the role of UAVs in safety management.

[0003] To solve these problems, some existing technologies consist of modules such as construction data collection and hazard source identification. By using optical flow technology and pyramid models to analyze the human movement trajectory, it is possible to accurately track and analyze the specific movement paths and behavior patterns of personnel at the construction site. By comparing the actual behavior trajectory with the preset safety distance, it is possible to judge whether personnel have entered a dangerous area to provide effective safety warnings. However, its degree of intelligence is limited, and it is impossible to achieve comprehensive intelligence in construction site safety management and difficult to flexibly adjust management strategies according to the collected data. On the other hand, the system functions are relatively single, it is difficult to integrate with other construction site management systems, and the ability to monitor complex environmental factors is weak, which cannot meet the requirements of improving the safety production level of construction sites. Summary of the Invention

[0004] In view of the above defects or improvement requirements of the prior art, the present invention provides an intelligent monitoring system and method for construction site safety risks of unmanned aerial vehicles integrated with deep learning. By using an improved YOLOv5 network incorporating an attention module, it can identify dangerous behaviors, prevent accidents, and reduce accident handling and compensation costs. At the same time, the UAV coordinates with each monitoring module to monitor dangerous risks, reduce the need for manual inspections, and significantly reduce long-term monitoring costs.

[0005] To achieve the above object, according to the first aspect of the present invention, there is provided an intelligent monitoring system for construction site safety risks of unmanned aerial vehicles integrated with deep learning, including a GPS positioning module, an air monitoring module, a hazard identification operation module, a voice alarm module, and a self-charging fixed inspection module;

[0006] The GPS positioning module generates three-dimensional coordinates and locates the UAV;

[0007] The air monitoring module monitors the concentration of particulate matter of a set substance in the air. When the particulate matter concentration exceeds the set value, it transmits a signal to the voice alarm module; when the particulate matter concentration in the air is not higher than the set value, the signal transmission stops.

[0008] The hazard identification operation module takes pictures of the construction site and determines whether there are hazardous behaviors; if there are hazardous behaviors, it transmits a signal to the voice alarm module, and when the hazardous behaviors disappear, the signal transmission stops.

[0009] After receiving the signal from the air monitoring module or the hazard identification operation module, the voice alarm module issues a voice alarm. At the same time, the voice alarm module obtains the three-dimensional coordinates of the hazard source through the GPS positioning module, and transmits the coordinates and the signal received from the air monitoring module or the hazard identification operation module to the supervisor.

[0010] Further, the GPS positioning module includes an infrared ranging module;

[0011] The infrared ranging module emits infrared rays, then converts the received infrared signal into an electrical signal, and uses to calculate the distances to each target. In the formula, L is the target distance, c is the speed of light, and t is the time required for the light pulse to travel back and forth.

[0012] Further, the GPS positioning module further includes a satellite positioning module, a precise positioning module, and an interference positioning module. The GPS positioning module fits different data obtained by the built-in modules to obtain positioning data;

[0013] The satellite positioning module simultaneously receives signals from several satellites and obtains a positioning result X through a fusion algorithm, where n is the total number of satellites, i is the satellite number, p(X|z i ) is the posterior probability of the positioning result X given the satellite signal measurement value z i , and p(z i ) is the prior probability synthesis signal of the satellite signal measurement value z i ;

[0014] The precise positioning module has RTK built-in and positions based on the principle of carrier phase measurement, In the formula, is the observed value, ρ is the distance from the satellite to the UAV, λ is the carrier wavelength, and N is the integer ambiguity;

[0015] The interference positioning module includes using an adaptive beamforming algorithm in an interference environment to adjust the antenna array weight vector w and adjust the signal-to-interference-plus-noise ratio SINR to achieve positioning; In the formula, w H is the conjugate transpose of w, and R Sis the correlation matrix between desired signals, R z is the correlation matrix of interference signals and noise.

[0016] Furthermore, the air monitoring module calculates the particle concentration of a set substance in the air based on laser scattering technology and generates a trend curve;

[0017] The method for calculating the particle concentration C is as follows:

[0018]

[0019] In the formula, I is the scattered light intensity actually received by the drone, I0 is the incident light intensity, m is the relative refractive index of the particle, V is the volume of a single particle, r is the distance between the drone and the scattering particle, and η is the wavelength of the incident light;

[0020] Among them, the gas uses pump suction sampling, and the dust uses diffusion sampling.

[0021] Furthermore, the hazard identification operation module includes an image acquisition and transmission module, a data preprocessing module, and a model construction and training module;

[0022] The image acquisition and transmission module takes pictures of the construction site image and transmits the original image to the data preprocessing module;

[0023] The data preprocessing module performs normalization, denoising, and dimension transformation processing on the original image to obtain a preprocessed image. The method of normalization is In the formula, p new is the pixel value after normalization processing, p old is the pixel value in the original image, p max is the maximum value that the pixel value can reach;

[0024] The model construction and training module uses an improved YOLOv5 network structure model incorporating an attention module and optimizes the model by adjusting the loss function L θ and optimizer parameters in real time.

[0025] Furthermore, the hazard identification operation module also includes a feature extraction and processing module and a result judgment and signal transmission module;

[0026] After obtaining the preprocessed image, the feature extraction and processing module generates a feature map through convolution operations and performs non-linear activation function processing:

[0027] F = Σ j K j ⊙I j ;

[0028] A(x) = max(0, x);

[0029] where \(F\) is the feature map generated after the convolution operation, \(j\) is the serial number of the convolution kernel, and \(K\) j is the \(j\)-th convolution kernel, \(I\) j is the local area of the input data, \(x\) is the output value of the convolution layer, and \(A(x)\) is the non-linear activation function; perform in-depth convolution processing:

[0030]

[0031] where \(L\) CLoU is the loss function of the matching index CLoU between the predicted box and the ground truth box, IoU is the intersection over union of the predicted box and the ground truth box, \(b, b\) gt are the center points of the predicted box and the ground truth box respectively, \(d\) is the Euclidean distance between the two center points, \(g\) is the diagonal distance of the minimum closed region of the predicted box and the ground truth box, \(v\) is the difference parameter of the aspect ratio of the predicted box and the ground truth box, and \(\alpha\) is the weight parameter for balancing the influence degree of \(v\) on CIoU; is the extracted feature vector, is the feature vector after normalization processing, is the Euclidean norm of the feature vector ; \(S\) is the cosine similarity, is the predefined standard feature vector, are respectively the Euclidean norms of.

[0032] Furthermore, the built-in algorithm of the self-charging fixed patrol module analyzes the distance between the drone and the charging station and the current battery status in real time; according to the set battery threshold, it autonomously judges whether the drone needs to return for charging; and sets a patrol path processing algorithm to record the daily patrol path situation and optimize the patrol path in real time.

[0033] According to the second aspect of the present invention, there is provided an intelligent monitoring method for the safety risks of a drone construction site integrated with deep learning, which is implemented by applying the integrated deep learning-based intelligent monitoring system for the safety risks of a drone construction site, and includes the following steps

[0034] S10, Battery judgment and charging processing. The self-charging fixed patrol module checks the battery before takeoff and during flight. If the battery is lower than the threshold, the drone automatically charges until the battery reaches the standard and then flies; if the battery reaches the standard, it flies directly;

[0035] S20, Multi-module collaborative monitoring during flight. The GPS positioning module generates three-dimensional coordinates and locates the drone; the air monitoring module monitors the particle concentration. When the particle concentration exceeds the set value, it enters S30; the danger recognition operation module collects images and judges dangerous behaviors. If there are dangerous behaviors, it enters S30;

[0036] S30. Voice alarm and information transmission. The voice alarm module receives signals and issues alarms, obtains the coordinates of the hazard source, and transmits the coordinates and the received signals to the supervisors.

[0037] Generally speaking, compared with the prior art by the above technical solutions conceived by the present invention, the following beneficial effects can be achieved:

[0038] 1. The monitoring system of the present invention uses an improved YOLOv5 network integrated with an attention module to identify dangerous behaviors, prevent accidents, and reduce accident handling and compensation costs. At the same time, the drone collaborates with each monitoring module to monitor dangerous risks, reduce the need for manual inspections, and significantly reduce long-term monitoring costs.

[0039] 2. The monitoring system of the present invention combines technologies such as infrared ranging and satellite positioning through the GPS positioning module to generate three-dimensional coordinates, ensuring accurate positioning and monitoring positions of the drones. The air monitoring module calculates the particle concentration and generates a trend curve based on laser scattering technology, improving the accuracy and efficiency of construction site safety monitoring.

[0040] 3. The monitoring of the present invention is linked with the system's hazard identification operation module and the voice alarm module to detect dangerous behaviors in real time and issue alarms, ensuring the safety of workers, enhancing their safety awareness, reducing construction accidents, and promoting the safety upgrade of the construction industry.

[0041] 4. The monitoring system of the present invention uses the built-in algorithm of the self-charging fixed inspection module to optimize the inspection path in real time, reduce the flight distance of the drones, lower energy consumption, further reduce operating costs, and automatically charge to maintain the sustainability of monitoring, improving the reliability of drone monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the system working provided by a preferred embodiment of the present invention;

[0043] Figure 2 It is a schematic diagram of the preprocessing of the input image for a multi-person scene provided by a preferred embodiment of the present invention;

[0044] Figure 3 It is a schematic diagram of the preprocessing of the input image for a single-person scene provided by a preferred embodiment of the present invention;

[0045] Figure 4 It is a schematic diagram of the user output display interface provided by a preferred embodiment of the present invention;

[0046] Figure 5 It is a schematic diagram of the flow of the intelligent monitoring method for construction site safety risks of drones integrated with deep learning provided by a preferred embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0048] Based on the above problems, the present invention integrates five key modules: a GPS positioning module, an air monitoring module, a hazard identification operation module, a voice alarm module, and a self-charging fixed inspection module, forming a comprehensive and automated construction site safety monitoring solution.

[0049] The GPS positioning module is the foundation of the system. It ensures that the drone can accurately locate specific areas of the construction site, achieving full coverage and precise monitoring of the construction area. Using infrared ranging, three-dimensional coordinates are generated and the data is transmitted to the background. Through high-precision GPS technology, the drone can automatically navigate to preset monitoring points and can also accurately record the location data of detected abnormal behaviors, facilitating subsequent viewing and review. Without manual operation, it greatly improves the convenience and accuracy of monitoring.

[0050] The air monitoring module focuses on the air quality of the construction site, especially the concentration of fine particulate matter (PM2.5). This module is crucial for protecting the health of workers because long-term exposure to high concentrations of PM2.5 increases the risk of respiratory diseases. By real-time monitoring the air quality, this module can timely remind the management to take measures, such as restricting operations, providing protective equipment, etc., to ensure the health and safety of workers.

[0051] The hazard identification operation module is the core of the system. It uses the YOLOv5 deep learning network to perform real-time analysis on the images captured by the drone to identify potential dangerous behaviors such as whether workers are wearing safety helmets, whether they are working at heights without correctly using safety belts, and whether they are working in dangerous areas. YOLOv5 has significant advantages in the field of object detection with its high speed and high accuracy, and can quickly identify and respond to abnormal situations on the construction site, greatly enhancing the initiative and timeliness of safety management.

[0052] The voice alarm module is immediately activated when a dangerous behavior is detected, sending the danger signal back to the remote control and issuing an alarm to remind the on-site workers to pay attention to safety. This immediate feedback mechanism can not only prevent potential accidents from occurring, but also enhance the safety awareness of workers and promote the construction of a safety culture.

[0053] The self-charging fixed inspection module ensures that the drone can automatically return to the charging station for charging when the battery level is low without manual intervention.

[0054] Please refer toFigure 1 , the present invention relates to an intelligent monitoring system for construction site safety risks of drones integrated with deep learning, including a GPS positioning module, an air monitoring module, a hazard identification operation module, a voice alarm module, and a self-charging fixed inspection module (system architecture); each module works collaboratively through data flow to ensure the accuracy of the judgment result.

[0055] Among them, in order to adapt to this system, the drone should have the following performance requirements:

[0056] Adopt an integrated flight platform, such as the Jingwei M30 series, which has the characteristics of rapid deployment, ultra-small volume, long endurance, and high protection level.

[0057] Support the integration of third-party hardware and APP, and provide cloud API, making the system capabilities more open and extensible.

[0058] It can realize 7×24-hour monitoring of the construction site, ensure safe production. Especially in linear engineering projects, drone inspections improve efficiency and reduce the labor intensity of personnel inspections.

[0059] It can automatically and efficiently collect engineering data, and through the cloud modeling function, quickly generate high-precision 2D and 3D models, providing an important basis for BIM-based analysis and application.

[0060] The GPS positioning module generates three-dimensional coordinates and locates the drone;

[0061] The air monitoring module monitors the concentration of set substance particles in the air. When the particle concentration exceeds the set value, it transmits a signal to the voice alarm module; when the particle concentration in the air is not higher than the set value, the signal transmission stops;

[0062] The hazard identification operation module takes pictures of the construction site and judges whether there are dangerous behaviors; if there are dangerous behaviors, it transmits a signal to the voice alarm module, and when the dangerous behavior disappears, the signal transmission stops;

[0063] After receiving the signal from the air monitoring module or the hazard identification operation module, the voice alarm module issues a voice alarm. At the same time, the voice alarm module obtains the three-dimensional coordinates of the hazard source through the GPS positioning module, and transmits the coordinates and the signal received from the air monitoring module or the hazard identification operation module to the supervisor.

[0064] 1. Implementation method of the GPS positioning module:

[0065] Establish a set of monitoring and identity recognition systems, which can capture the RGB color images and infrared images on-site, and then transmit these image materials to the identity recognition module via the network for subsequent analysis and processing.

[0066] First, infrared rays are emitted through an infrared emission circuit, and these rays are emitted at a certain frequency and angle. The emitted infrared light is reflected back after encountering an obstacle. The intensity of the reflected light changes with the distance of the object. When the distance is close, the reflected light intensity is strong, and when the distance is far, the reflected light is weak. The reflected infrared rays are received by the infrared receiving circuit of the system. The received infrared signal is converted into an electrical signal and amplified by an amplifier for subsequent processing. Using

[0067]

[0068] the distance is calculated, where c is the speed of light, L is the target distance, and t is the time required for the light pulse to travel back and forth.

[0069] Multiple infrared sensors are installed on the drone, and each sensor is responsible for measuring the distance in one direction. The collected distance data is processed through data fusion technology, combined with the flight data of the drone (such as GPS position, IMU attitude data, etc.) to determine the precise position of the object in three-dimensional space, and the measured distance data is converted into points in a three-dimensional coordinate system. The final three-dimensional position information can be used for various applications, such as obstacle avoidance, dangerous behavior positioning, map construction, etc. These information are transmitted to the ground system through a wireless network for further processing and analysis.

[0070] The GPS positioning module includes an infrared distance measurement module;

[0071] The infrared distance measurement module emits infrared rays, then converts the received infrared signal into an electrical signal, and uses to calculate the distances to each target. In the formula, L is the target distance, c is the speed of light, and t is the time required for the light pulse to travel back and forth.

[0072] The GPS positioning module further includes a satellite positioning module, a precise positioning module, and an interference positioning module. The GPS positioning module fits different data obtained by the built-in modules to obtain positioning data;

[0073] The satellite positioning module simultaneously receives signals from several satellites and obtains a positioning result X through a fusion algorithm, where n is the total number of satellites, i is the satellite number, p(X|z i ) is the posterior probability of the positioning result X given the satellite signal measurement value z i , and p(z i ) is the prior probability integrated signal of the satellite signal measurement value z i ;

[0074] The precise positioning module has RTK built-in and locates according to the carrier phase measurement principle, In the formula, is the observed value, ρ is the distance between the satellite and the UAV, λ is the carrier wavelength, and N is the integer ambiguity;

[0075] The interference positioning module includes using an adaptive beamforming algorithm in an interference environment to adjust the antenna array weight vector w and adjust the signal-to-interference-plus-noise ratio (SINR) to achieve positioning; In the formula, w H is the conjugate transpose of w, R S is the correlation matrix between desired signals, R z is the correlation matrix of interference signals and noise.

[0076] 2. Implementation method of the air monitoring module

[0077] This system designs an airborne air quality monitoring device integrated on the UAV. Taking the monitoring of PM2.5 as an example, the device is embedded with PM2.5 and PM10 sensors to collect the concentration of particulate matter in the air in real time. The system also includes a detailed detection database, which contains detailed data on meteorological conditions and air quality. By flying the UAV with this monitoring device to a specific area, the concentration of PM2.5 can be accurately measured, and the measured real-time data can be transmitted back to the ground control station through 2.4G wireless communication technology to display the concentration change trend in a graphical way. At the same time, the monitoring data will be saved synchronously on the UAV and the ground control station for subsequent data analysis and recording.

[0078] In addition, the data tables in the detection database support automatic data import or manual editing through client software. Such a design makes data management more efficient and convenient. Users can directly import data through the USB interface or update and maintain the information in the database according to needs to ensure that the information in the database is always up-to-date and accurate. This integrated monitoring solution not only improves the flexibility and timeliness of air quality monitoring, but also provides strong data support for environmental management and decision-making.

[0079] The air monitoring module calculates the particle concentration of a set substance in the air based on laser scattering technology and generates a trend curve;

[0080] The method for calculating the particle concentration C is:

[0081]

[0082] In the formula, I is the scattered light intensity actually received by the UAV, I0 is the incident light intensity, m is the relative refractive index of the particle, V is the volume of a single particle, r is the distance between the UAV and the scattering particle, and η is the incident light wavelength;

[0083] Among them, the gas uses pump suction sampling, and the dust uses diffusion sampling.

[0084] 3. Implementation Modes of the Hazard Identification Module

[0085] Please refer to Figure 2 and Figure 3 , in some preferred embodiments, a high-definition camera is installed on the drone to capture images of the construction site during the inspection and transmit them back to the YOLOv5 network for identification. The image size is 1920*1080 pixels to provide sufficient details for the identification algorithm to analyze.

[0086] Denoise, normalize, and crop the original image: Scale the original image to the target size of 640*640 pixels, perform an affine transformation on the image to adjust it to the target size, convert the image from BGR to RGB, and normalize the pixel values of the image from [0, 255] to [0, 1] by dividing each pixel value by 255. Then transform the dimension of the image from height, width, channels (H, W, C) to channels, height, width (C, H, W), and add a batch dimension (B) at the front of the next data in the figure, transforming it from C, H, W to B, C, H, W.

[0087] Send the input data into the network, apply the convolutional kernel (filter) to slide on the input data, calculate the dot product of the convolutional kernel and the local area of the input data, and generate the feature map. Apply the non-linear activation function ReLU to the output of the convolutional layer.

[0088] Use multiple layers of convolution, repeat the convolution operation, and use different numbers and different sizes of convolutional kernels for each layer to extract features of different scales, and apply a non-linear activation function after each convolutional layer. As the network depth increases, the features of lower levels are gradually combined into higher-level features.

[0089] In the CNN structure of this design, a fully connected layer is linked after the convolutional layer, the feature map is flattened into a one-dimensional vector, and then the classification task is performed through the fully connected layer.

[0090] For the difference between the detection result and the true result, use the CLoU loss function to quantify this difference

[0091]

[0092] where b and b gt respectively represent the center points of the predicted box and the true box, ρ represents the Euclidean distance between the two center points, and c represents the diagonal distance of the minimum enclosing region of the predicted box and the true box.

[0093] In the YOLOv5 network, the FPN structure is adopted to achieve high-dimensional mapping and is constructed through the CSP structure.

[0094] The FPN structure captures strong semantic features in a top - down manner, fuses feature maps of different scales. It first performs a sampling operation on the smaller feature maps and then superimposes them on the larger - sized feature maps, introducing other information such as illumination changes and background noise, solving the scale problem in detection, and increasing the depth and robustness of the network.

[0095] After FPN, the PAN structure is used to strengthen feature aggregation. The feature maps are resized to a smaller size by performing a bit - wise addition operation on two feature maps of the same size.

[0096] By combining FPN and PAN, the function of target localization is more effectively completed.

[0097] The extracted feature vectors are normalized by dividing the feature vectors by the L2 norm to make their length 1.

[0098] The cosine similarity distance metric is used to calculate the distance between the feature vector and the predefined standard safety - helmet feature vector. According to the distance between the feature vectors, a similarity score is calculated. The closer the cosine value is to 1, the higher the similarity.

[0099] The sigmoid function is used to output the result of danger recognition. The cosine similarity metric is used to compare the feature vector of the detected image with the feature vectors in the database. It is determined whether there is a dangerous behavior in the image. If so, the signal is transmitted to the safety alarm module to remind the management personnel to intervene.

[0100] The danger recognition operation module includes an image acquisition and transmission module, a data pre - processing module, and a model construction and training module;

[0101] The image acquisition and transmission module takes pictures of the construction site image and transmits the original image to the data pre - processing module;

[0102] The data pre - processing module performs normalization, denoising, and dimension transformation on the original image to obtain a pre - processed image. The method of normalization is where p new is the pixel value after normalization processing, p old is the pixel value in the original image, and p max is the maximum value that the pixel value can reach;

[0103] The model construction and training module adopts an improved YOLOv5 network structure model incorporating an attention module, and optimizes the model by adjusting the loss function L θ and optimizer parameters in real - time.

[0104] The danger recognition operation module also includes a feature extraction and processing module and a result judgment and signal transmission module;

[0105] After obtaining the preprocessed image, the feature extraction and processing module generates a feature map through convolution operation and performs non-linear activation function processing:

[0106] F = Σ j K j ⊙I j ;

[0107] A(x) = max(0, x);

[0108] In the formula, F is the feature map generated after convolution operation, j is the serial number of the convolution kernel, K j is the j-th convolution kernel, I j is the local area of the input data, x is the output value of the convolution layer, A(x) is the non-linear activation function; perform in-depth convolution processing:

[0109]

[0110] In the formula, L CLoU is the loss function of the matching index CLoU between the predicted box and the ground truth box, IoU is the intersection over union of the predicted box and the ground truth box, b, b gt are the center points of the predicted box and the ground truth box respectively, d is the Euclidean distance between the two center points, g is the diagonal distance of the minimum closed area of the predicted box and the ground truth box, v is the difference parameter of the aspect ratio of the predicted box and the ground truth box, and α is the weight parameter for balancing the influence degree of v on CIoU; is the extracted feature vector, is the feature vector after normalization processing, is the Euclidean norm of the feature vector ; S is the cosine similarity, is the predefined standard feature vector, are respectively the Euclidean norms of

[0111] 4. Implementation method of the voice alarm module

[0112] On a construction site, personal protective equipment such as workers' safety helmets and safety belts is equipped with electronic tags and their communication interfaces, and these interfaces are connected to the alarm system. The YOLOv5 network in the control center analyzes the on-site situation to determine whether there are potential safety hazards. Once a potential dangerous behavior is detected, the system sends a warning signal to the electronic tag on the protective equipment of the violating worker through radio frequency technology. Subsequently, these warning signals are transmitted to the ground console or the operator's remote control through the communication port of the electronic tag, and the operator further issues an alarm.

[0113] The alarm information and location information are transmitted to the alarm unit based on the electronic tag transmission interface. When the alarm unit sends out an alarm, it broadcasts the location information of the safety helmet to provide accurate and effective warnings about dangerous behaviors.

[0114] 5. Implementation Modes of the Self-service Charging and Fixed Patrol Module

[0115] The intelligent unmanned aerial vehicle (UAV) charging and fixed patrol module consists of a UAV, a charger, a base, and a power supply unit, and integrates a power monitoring and a monitoring module to capture relevant data on the patrol path and area. When the UAV lands, it will be placed on the platform of the base. Before performing the patrol task, the UAV will send its flight route information to the control system. The power monitoring module is responsible for collecting the current power of the UAV and reporting it to the control system.

[0116] When the UAV is ready and at a predetermined take-off point, the system will check its power status. If the UAV has not taken off and has sufficient power, reaching or exceeding the set threshold, the control system will generate an instruction to start the UAV's flight task. On the contrary, if the UAV has insufficient power, below the set threshold, the system will not only prevent the UAV from taking off but also issue a charging prompt to guide the operator to perform the charging operation to ensure that the UAV can perform tasks in a safe and effective state. Through this intelligent power management and patrol path planning, the UAV can complete the patrol work more efficiently and safely.

[0117] The self-service charging and fixed patrol module internally sets an algorithm to analyze the distance between the UAV and the charging station and the current power status in real time; based on the set power threshold, it independently judges whether the UAV needs to return for charging; and sets a patrol path processing algorithm to record the daily patrol path situation and optimize the patrol path in real time.

[0118] 6. Output Interface and Report

[0119] Please refer to Figure 4 , an intuitive user interface should be designed so that the staff can intuitively view information such as the on-site images taken by the high-definition camera of the area patrolled by the UAV, the real-time GPS positioning data, the monitoring data of the concentration of harmful substance particles in the air, the alarm status, the power of the UAV, and the patrol route. The detection data should also be saved in the background to facilitate the staff to statistically analyze and inspect the construction status of the construction site.

[0120] In summary, the model architecture of this system:

[0121] Convolutional Layer: Multiple convolutional layers are used to extract the features of the input data. Each convolutional layer includes several convolutional kernels that can identify different features in the data.

[0122] Pooling layer: The pooling layer is used to reduce the spatial dimension of the data, reduce the computational complexity, and enhance the feature expression ability.

[0123] Fully connected layer: Flatten the features extracted by the convolutional and pooling layers and perform further processing through the fully connected layer to obtain the final safety monitoring result.

[0124] Output layer: Use the sigmoid activation function to output the probability values of multiple categories, indicating different helmet wearing conditions and seat belt usage conditions.

[0125] Risk assessment module: Use the risk assessment model based on QPSO-BP to calculate the risk of safety behaviors.

[0126] Obtain the worker's work image through a camera, perform histogram equalization processing and data augmentation processing on the image to obtain an updated work image;

[0127] Extract key feature points in the image through the preprocessing system, and crop the key feature points to obtain a cropped area;

[0128] Judge whether the cropped area is an image of a safety helmet and an image of wearing a seat belt, classify it into two categories, and process it in the safety helmet recognition module and the seat belt recognition module respectively;

[0129] When the cropped area is an image of a safety helmet, use a preset convolutional neural network to extract the first regional feature vector of the unobstructed area, compare it with the feature vector of the staff's face image already entered in the system database, and judge the identity of the staff;

[0130] Output the corresponding name information of the staff through the variational autoencoder (VAE);

[0131] When the cropped area is an image of the safety helmet area, use a convolutional neural network to extract the second regional feature vector of the safety helmet area;

[0132] Compare the second regional feature vector with the existing standard safety helmet feature vector to detect whether the inspector is wearing the safety helmet correctly;

[0133] Use the sigmoid function to output the safety helmet wearing status, including wearing, not wearing, and improper wearing;

[0134] Calculate the fuzzy set of this face recognition by combining the face recognition result, the safety helmet detection result and the risk assessment result;

[0135] When the cropped area is a safety belt image, use a preset aggregation neural network to extract the feature vector of the safety belt area, and compare it with the existing safety belt feature vectors in the database to determine whether the worker is using the safety belt correctly.

[0136] The input end of this design includes Mosaic data augmentation, adaptive anchor box calculation, and adaptive image scaling;

[0137] The Backbone structure of this design includes a Focus structure and a CSP structure. The attention system can accurately identify the safety helmet image and the safety belt image respectively, which helps with feature extraction;

[0138] After classifying and identifying the dangerous behaviors of workers, transmit the danger signal to the remote controller, and let the controller decide to remind the workers at the construction site. And the drone is integrated with a GPS positioning module, an air monitoring module, and a self-charging fixed inspection module, which can transmit accurate three-dimensional position positioning information when transmitting the dangerous behavior to the remote control end, transmit the air quality regularly, or transmit back the data and remind the controller when detecting abnormal air substance data for processing, to ensure the safe operation of workers and protect the public environment of the city. In addition, the drone automatically returns to the charging point for charging after low battery or after the inspection is completed. The controller can set the inspection route and time of the drone, and the drone can automatically inspect the construction site without the need for a dedicated person to remotely control it.

[0139] As Figure 5 shown, as another aspect of the present invention, it also relates to an intelligent monitoring method for construction site safety risks of drones integrated with deep learning, including the following steps:

[0140] S10, Battery level judgment and charging process. The self-service charging and fixed inspection module checks the battery level before takeoff and during flight. If the battery level is lower than the threshold, the UAV automatically charges until the battery level reaches the standard and then takes off; if the battery level reaches the standard, it takes off directly. The self-service charging and fixed inspection module ensures the endurance of the UAV and realizes automatic inspection. In this process, the automatic inspection of the UAV and energy management are combined. Through the preset inspection route and schedule, the UAV can automatically execute flight tasks, perform data collection, and automatically download and synthesize the collected data into orthophotos and 3D models. The system includes wireless communication technology and GPS positioning system to assist in finding the charging platform and docking the charging interface, realizing automatic return and charging when the UAV's battery is low. The charging module supports both wireless and wired charging methods to adapt to different inspection requirements and scenarios. The fully automatic hangar provides storage and charging for the UAV, guides the UAV to land through the positioning module and conducts automatic charging. In addition, the system is designed with Mecanum wheels to enable the UAV to adjust its position to adapt to the positioning of the positioning module and ensure accurate docking for charging. Through this design, the UAV can achieve fully automatic inspection and endurance, greatly reducing labor costs and improving the efficiency of inspection work and safety management level.

[0141] S20, Multi-module collaborative monitoring during flight. The GPS positioning module generates three-dimensional coordinates and locates the UAV; the air monitoring module monitors the particle concentration. When the particle concentration exceeds the set value, it enters S30; the danger recognition operation module collects images and judges dangerous behaviors. If there are dangerous behaviors, it enters S30; the GPS positioning module ensures the accurate positioning of the UAV and monitors the construction site status in real time; the PM2.5 monitoring module evaluates the air quality to protect the health of workers; the danger recognition operation module uses the YOLOv5 algorithm to analyze image data and identify potential dangerous behaviors; the voice alarm module issues a warning when detecting dangerous behaviors to timely remind the on-site personnel.

[0142] S30, Voice alarm and information transmission. The voice alarm module receives the signal and issues an alarm, obtains the coordinates of the hazard source, and transmits the coordinates and the received signal to the supervisors. In this process, the system will timely display the alarm information on the user interface and record the time and type of the alarm for subsequent safety management and accident traceability. A function of transmitting the alarm signal back to the remote controller through wireless communication is designed, enabling the operator to receive the alarm information in real time and take corresponding emergency measures. A set of alarm response mechanisms, including audible and visual alarms, information push, etc., are designed to ensure that when dangerous behaviors are detected, the attention of on-site personnel and safety officers can be quickly attracted.

[0143] Those skilled in the art can easily understand that the above description is only a preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent monitoring system for construction site safety risks of drones integrated with deep learning, characterized in that, It includes a GPS positioning module, an air monitoring module, a hazard identification operation module, a voice alarm module, and a self-charging fixed patrol module; The GPS positioning module generates three-dimensional coordinates and locates the drone; The air monitoring module monitors the concentration of set substance particles in the air. When the particle concentration exceeds the set value, it transmits a signal to the voice alarm module; when the particle concentration in the air is not higher than the set value, the signal transmission stops; The hazard identification operation module takes pictures of the construction site image and judges whether there are dangerous behaviors; if there are dangerous behaviors, it transmits a signal to the voice alarm module, and after the dangerous behaviors disappear, the signal transmission stops; After receiving the signal from the air monitoring module or the hazard identification operation module, the voice alarm module issues a voice alarm. At the same time, the voice alarm module obtains the three-dimensional coordinates of the hazard source through the GPS positioning module, and transmits the coordinates and the signal received from the air monitoring module or the hazard identification operation module to the supervisor.

2. The intelligent monitoring system for drone construction site safety risks integrated with deep learning according to claim 1, characterized in that, The GPS positioning module includes an infrared ranging module; The infrared ranging module emits infrared rays and then converts the received infrared signals into electrical signals. By using the distances to each target are calculated. In the formula, L is the target distance, c is the speed of light, and t is the time required for the optical pulse to travel back and forth.

3. The intelligent monitoring system for drone construction site safety risks integrated with deep learning according to claim 2, wherein The GPS positioning module further includes a satellite positioning module, a precise positioning module, and an interference positioning module. The GPS positioning module fits different data obtained by the built-in modules to obtain positioning data; The satellite positioning module simultaneously receives signals from several satellites and obtains the positioning result X through a fusion algorithm. where n is the total number of satellites, i is the satellite serial number, p(X|z i ) is the posterior probability of the positioning result X given the satellite signal measurement value z i , p(z i ) is the prior probability synthesis signal of the satellite signal measurement value z i . The precise positioning module is built with RTK and locates based on the principle of carrier phase measurement. In the formula, is the observed value, ρ is the distance from the satellite to the UAV, λ is the carrier wavelength, and N is the integer ambiguity. The interference localization module includes implementing localization in an interference environment using an adaptive beamforming algorithm, and adjusting the signal-to-interference-plus-noise ratio (SINR) by adjusting the antenna array weight vector w. In the formula, w H is the conjugate transpose of w, and R S is the correlation matrix between desired signals, and R z is the correlation matrix of interference signals and noise.

4. The intelligent monitoring system for unmanned aerial vehicle construction site safety risks integrated with deep learning according to claim 3, wherein, The air monitoring module calculates the particle concentration of the set substance in the air based on the laser scattering technology and generates a trend curve; The method for calculating the particle concentration C is as follows: In the formula, I is the scattered light intensity actually received by the drone, I0 is the incident light intensity, m is the relative refractive index of the particle, V is the volume of a single particle, r is the distance between the drone and the scattering particle, and η is the wavelength of the incident light; Among them, the gas adopts pump suction sampling, and the dust adopts diffusion sampling.

5. The intelligent monitoring system for construction site safety risks of an unmanned aerial vehicle integrated with deep learning according to claim 4, characterized in that, The hazard identification operation module includes an image acquisition and transmission module, a data preprocessing module, and a model construction and training module; The image acquisition and transmission module takes pictures of the construction site image and transmits the original image to the data preprocessing module; The data preprocessing module performs normalization, denoising, and dimensional transformation on the original image to obtain a preprocessed image, where the normalization method is In the formula, p new is the pixel value after normalization processing, p old is the pixel value in the original image, p max is the maximum value that the pixel value can reach; The model construction and training module adopts an improved YOLOv5 network structure model integrated with an attention module, and optimizes the model by adjusting the loss function L θ and the optimizer parameters in real time.

6. The intelligent monitoring system for construction site safety risks of an unmanned aerial vehicle integrated with deep learning according to claim 5, wherein, The hazard identification operation module further includes a feature extraction and processing module and a result judgment and signal transmission module; After obtaining the preprocessed image, the feature extraction and processing module generates a feature map through convolution operation and performs a non-linear activation function processing: F = ∑ j K j ⊙I j ; A(x) = max(0, x); where F is the feature map generated after the convolution operation, j is the convolution kernel index, K j is the j-th convolution kernel, I j is the local region of the input data, x is the output value of the convolutional layer, and A(x) is the non-linear activation function; perform in-depth convolution processing: Where L CLoU is the loss function of the matching metric CLoU between the predicted box and the ground truth box, IoU is the intersection over union of the predicted box and the ground truth box, b, b gt are the center points of the predicted box and the ground truth box respectively, d is the Euclidean distance between the two center points, g is the diagonal distance of the minimum enclosing region of the predicted box and the ground truth box, v is the difference parameter of the width-to-height ratio between the predicted box and the ground truth box, and α is the weight parameter for balancing the influence degree of v on CIoU; is the extracted feature vector, is the feature vector after normalization processing, is the feature vector 's Euclidean norm; S is the cosine similarity, is the predefined standard feature vector, are respectively 's Euclidean norm.

7. The intelligent monitoring system for construction site safety risks of an unmanned aerial vehicle integrated with deep learning according to claim 6, characterized in that, The self-charging fixed patrol module internally sets an algorithm to analyze the distance between the drone and the charging station and the current power status in real time; according to the set power threshold, it autonomously judges whether the drone needs to return for charging; and sets a patrol path processing algorithm to record the daily patrol path conditions and optimize the patrol path in real time.

8. An intelligent monitoring method for the safety risks of a drone construction site integrated with deep learning, characterized in that it is implemented by using the intelligent monitoring system for the safety risks of a drone construction site integrated with deep learning as described in any one of claims 1-7, and includes the following steps: S10, power judgment and charging processing. The self-charging fixed patrol module checks the power before takeoff and during flight. If the power is lower than the threshold, the drone automatically charges until the power reaches the standard and then flies; if the power reaches the standard, it flies directly; S20, Multi-module collaborative monitoring during flight. The GPS positioning module generates three-dimensional coordinates and locates the UAV; the air monitoring module monitors the particle concentration, and when the particle concentration exceeds the set value, it enters S30; the danger identification operation module collects images and judges dangerous behaviors. If there are dangerous behaviors, it enters S30; S30, Voice alarm and information transmission. The voice alarm module receives the signal and issues an alarm, obtains the coordinates of the hazard source, and transmits the coordinates and the received signal to the supervisor.

Images