High-altitude operation safety space detection method and system

Abstract

The invention discloses a high-altitude operation safety space detection method and system. The system comprises a radar monitoring module, a video monitoring module, a data processing module, an alarm module and a data server. The radar monitoring module is responsible for collecting three-dimensional space data of a working area, the video monitoring module collects video images, the data processing module carries out data preprocessing and fusion, the alarm module triggers alarm when abnormity is detected, and the data server stores and processes data. The radar monitoring module adopts a Livox radar, and the video monitoring module adopts a Haikangwei video monitoring camera; and the data processing module adopts a Ubuntu operating system for processing. According to the invention, safety monitoring and early warning can be automatically carried out, the burden of monitoring personnel is reduced, and the operation safety is improved.

Description

Technical Field

[0001] The present invention relates to the field of high-altitude operation detection, and in particular to a high-altitude operation safety space detection method and system. Background Art

[0002] The field of high-altitude work safety monitoring technology has developed rapidly with the acceleration of industrialization and urbanization. This field mainly involves the use of advanced monitoring technology to ensure the safety of high-altitude workers and reduce the risk of accidents. Technical means include video monitoring and radar monitoring. With the advancement of technology, these monitoring technologies have played an important role in improving work safety, but there are also some limitations and shortcomings.

[0003] In the field of aerial work safety monitoring, the Thunderball series products of Company A and the products of Company B that integrate millimeter-wave radar are the more advanced technology representatives on the market. The Thunderball series products of Company A integrate radar and video monitoring technology, which can achieve all-weather and long-distance monitoring. However, these products have also exposed some defects in actual application.

[0004] Defects and shortcomings of the existing technology:

[0005] Large size: Leiqiu products and products that integrate millimeter-wave radars are often large in size, which is not conducive to installation and deployment in complex or narrow high-altitude working environments. In addition, millimeter-wave radars need to be recalibrated after each installation before they can be used accurately, which is contrary to the application scenario of frequent changes in construction sites during high-altitude construction.

[0006] Difficulty in secondary development: Existing products usually do not allow users to perform customized secondary development. The core functional permissions are not open to users and only provide users with usage permissions, which limits the flexibility and scalability of the system and cannot meet users' customization needs.

[0007] High false alarm rate: In fusion perception technology, due to the limitations of algorithm filtering functions, existing technologies may have a high false alarm rate when detecting targets, especially in scenarios of multi-target tracking and monitoring.

[0008] Low technical integration: Existing technical solutions often fail to effectively integrate the advantages of radar and video surveillance, resulting in performance bottlenecks in target detection and tracking. Summary of the invention

[0009] The technical problem to be solved by the present invention is: how to solve the problems of large volume, difficulty in secondary development and high false alarm rate in the existing technology, and provide a method and system for detecting a safe space for aerial work.

[0010] The present invention solves the above technical problems through the following technical solutions, which include a radar monitoring module, a video monitoring module, a data processing module, an alarm module and a data server;

[0011] The radar monitoring module is used to collect three-dimensional spatial data of the operating area;

[0012] The video monitoring module is used to collect video images;

[0013] The data processing module is used for data preprocessing and fusion;

[0014] The alarm module is used to trigger an alarm when an abnormality is detected;

[0015] Data servers are used to store and process data.

[0016] Furthermore, the radar monitoring module adopts Livox radar;

[0017] The video surveillance module adopts Hikvision surveillance camera;

[0018] The data processing module uses the Ubuntu operating system for processing.

[0019] A method for detecting a safe space for high-altitude operations, the method comprising the following steps:

[0020] Step 1: The radar monitoring module is started to collect data;

[0021] Step 2: The video surveillance module is started to collect data;

[0022] Step 3: The data preprocessing module preprocesses the received data;

[0023] Step 4: The data processing module independently processes the lidar point cloud and video stream. It includes two branches. One branch extracts point cloud features to predict 3D information, and the other branch extracts video image features to predict 3D information. The features of both are then projected into a unified BEV (bird's eye view) space, where they are fused and the deep learning model is used for object detection.

[0024] Step 5: Assign corresponding weights to the judgment results of the video and the judgment results of the laser radar. When the judgment condition is triggered, it is considered that non-operating personnel or other objects have entered the working area, and it is determined that the construction personnel are unsafe at this time. If not, return to step 4, if yes, proceed to the next step;

[0025] Step 6: The alarm module issues an alarm through email and warning prompts on the web interface, and records the alarm event;

[0026] Step 7: If you need to continue monitoring, return to step 3, otherwise end the monitoring.

[0027] Furthermore, the specific process of step 1 is as follows: start the robot operating system (ROS) on the edge computing platform, and then start the ROS driver of the livox laser radar, which is officially provided by livox; by configuring the corresponding ROS topic configuration file, the point cloud data and the corresponding data are published in the form of a topic, and the websocket function is used to enable the web end to receive the point cloud data of the radar; after successful startup, the Livox laser radar starts to scan the working area and collect three-dimensional space data.

[0028] Furthermore, the specific process of step 2 is as follows: using the camera software development kit (Software Development Kit) of the video monitoring module to configure the operating environment of the camera, so that the program can find the relevant dependent packages; filling in the static address IP, user, password and open port of the camera, so as to obtain the real-time camera picture; the video monitoring module starts to capture the video image of the working area.

[0029] Furthermore, the specific contents of the object detection algorithm used in the object detection process in step 4 are as follows:

[0030] I: Image preprocessing: denoising and normalizing the images in the video stream;

[0031] II: Feature Extraction: Extract key features from images using deep learning models;

[0032] III: Object Recognition: Match the extracted features with the categories in the pre-trained YOLOv8 model to identify objects;

[0033] IV: Confidence assessment: Conduct confidence assessment on the recognition results to determine the accuracy of the recognition;

[0034] V: Threshold judgment: If the confidence is higher than the preset threshold, the detected object is considered to be a real target, otherwise it is ignored;

[0035] VI: Alarm decision: Determine whether to trigger an alarm based on the detection results and safety rules.

[0036] Furthermore, the image preprocessing in step I includes histogram equalization and Gaussian blur;

[0037] The feature extraction in step II is performed using multiple layers of convolutional neural networks;

[0038] The threshold in step V is dynamically adjusted according to the actual application scenario and security requirements;

[0039] The alarm decision in step VI takes into account historical alarm data and real-time environmental factors.

[0040] Compared with the prior art, the present invention has the following advantages: the method and system for detecting safe space for aerial work provide a device and system for detecting safe space for aerial work that is compact, easy for secondary development, highly adaptable to the environment, has a low false alarm rate, high technical integration, reasonable cost, and simplified structure. The system combines Livox radar and Hikvision surveillance cameras, uses edge computing platforms and cloud servers for data processing, and realizes real-time monitoring and safety warning of aerial work areas. It also supports warning prompts on emails and web interfaces to improve the practicality and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a structural schematic diagram of the high-altitude operation safety space detection system of the present invention;

[0042] Figure 2 is a schematic diagram of a video detection algorithm of the present invention;

[0043] Figure 3 Schematic diagram of the radar detection algorithm of the present invention. DETAILED DESCRIPTION

[0044] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0045] like Figure 1 As shown, this embodiment provides a technical solution: a high-altitude work safety space detection system, including a radar monitoring module, a video monitoring module, a data processing module, an alarm module and a data server. The radar monitoring module is responsible for collecting three-dimensional spatial data of the work area, the video monitoring module collects video images, the data processing module performs data preprocessing and fusion, the alarm module triggers an alarm when an abnormality is detected, and the data server stores and processes data.

[0046] The radar monitoring module uses Livox radar, the video monitoring module uses Hikvision surveillance camera, and the data processing module uses Ubuntu operating system for processing.

[0047] The image data lacks depth information and cannot locate the target, and the camera is easily affected by illumination changes, resulting in low algorithm tracking accuracy. The lidar point cloud has rich three-dimensional point cloud spatial information, but lacks color information and is sparse. When the detection target is far away, the effect of the target detection algorithm based on deep learning is significantly reduced. Therefore, the present invention combines the advantages of image category information and point cloud distance information to effectively make up for each other's shortcomings and effectively realize target detection and tracking. The first purpose of the present invention is to provide a high-altitude operation safety space detection system that can automatically perform safety monitoring and early warning, reduce the burden on monitoring personnel, and improve operation safety.

[0048] A method for detecting a safe space for high-altitude operations comprises the following steps:

[0049] 1. Start the radar monitoring module: Start the Robot Operating System (ROS) on the edge computing platform, and then start the ROS driver of the Livox LiDAR, which is provided by Livox. By configuring the corresponding ROS topic configuration file, the point cloud data and the corresponding data are published in the form of a topic, and the websocket function is used to enable the web end to receive the point cloud data of the radar. After successful startup, the Livox LiDAR starts to scan the operating area and collect three-dimensional spatial data;

[0050] 2. Start the video surveillance module: Use the camera software development kit (Software Development Kit) provided by Hikvision to configure the camera's operating environment so that the program can find related dependency packages. Fill in the camera's static address IP, user, password and open port to obtain real-time camera images. Hikvision surveillance cameras begin to capture video images of the operating area;

[0051] 3. The data preprocessing module preprocesses the received data: including filtering of radar point cloud data, downsampling and denoising of video images, normalization, format conversion and color channel conversion to facilitate the use of subsequent steps;

[0052] 4. The data processing module independently processes the lidar point cloud and video stream. One branch extracts point cloud features to predict 3D information, and the other branch extracts video image features to predict 3D information. The features of both are then projected into a unified BEV (bird's eye view) space, where they are fused and the deep learning model is used for object detection.

[0053] 5. The judgment result of the video and the judgment result of the laser radar are assigned corresponding weights. If the judgment condition is triggered, it is considered that non-operating personnel or other objects have entered the working area, and it is determined that the construction personnel are unsafe at this time. If not, return to step 4. If yes, proceed to the next step;

[0054] 6. The alarm module issues alarms via email and warning prompts on the web interface, and records alarm events;

[0055] 7. If you need to continue monitoring, return to step 3, otherwise end monitoring.

[0056] The object detection algorithm in step 4 is specifically:

[0057] I. Image preprocessing: denoising and normalizing images in the video stream;

[0058] II. Feature extraction: Use deep learning models to extract key features from images;

[0059] III. Object Recognition: Match the extracted features with the categories in the pre-trained YOLOv8 model to identify objects;

[0060] IV. Confidence assessment: Conduct confidence assessment on the recognition results to determine the accuracy of the recognition;

[0061] V. Threshold judgment: If the confidence is higher than the preset threshold, the detected object is considered to be a real target, otherwise it is ignored;

[0062] VI. Alarm decision: Determine whether to trigger an alarm based on the detection results and safety rules.

[0063] The image preprocessing in step I includes histogram equalization and Gaussian blur.

[0064] The feature extraction in step II is performed using multiple layers of convolutional neural networks.

[0065] The threshold in step V is dynamically adjusted according to the actual application scenario and security requirements.

[0066] The alarm decision in step VI takes into account historical alarm data and real-time environmental factors.

[0067] Through the above technical solution, the present invention can realize real-time monitoring and safety warning of high-altitude operation areas, thereby improving the safety and efficiency of operations.

[0068] The main technical principles of the above process are:

[0069] 1. YOLOv8 target detection principle: Based on the YOLO series, it introduces a number of innovations, including a new backbone network, Anchor-Free detection head, and a new loss function to improve performance and flexibility. YOLOv8 adopts the Anchor-Free method, so the anchors parameter is removed in the configuration file, and multiple different versions of model parameters are supported. The head part of YOLOv8 changes from a coupled head to a decoupled head structure, separating the classification and regression branches, and using the integral form representation proposed in the Distribution Focal Loss strategy for the regression branch. This representation transforms the coordinate prediction into a distribution, thereby improving the accuracy of positioning. In terms of Loss calculation, YOLOv8 adopts the TaskAlignedAssigner positive sample allocation strategy and introduces Distribution Focal Loss. The matching strategy can be expressed as:, where is the predicted score corresponding to the labeled category, is the loU of the predicted box and the gt box, and the two are multiplied to measure the degree of alignment.

[0070] 2. The BEVFusion algorithm, as the core of the radar detection algorithm, is a method that fuses visual information and radar point cloud data for 3D target detection. This algorithm is particularly suitable for aerial work safety space detection systems, and can convert data captured from different perspectives into a unified bird's-eye view (BEV) representation for more effective target detection and safety monitoring. The key steps of the BEVFusion algorithm include independently processing the three-dimensional point cloud data collected by the radar monitoring module and the video image data captured by the video monitoring module, and then projecting these data into the BEV space. This process can be expressed by the following formula:

[0071] Among them, represents the three-dimensional point cloud data collected by the radar monitoring module, represents the video image data captured by the video monitoring module, represents the data projected onto the two-dimensional BEV plane, and the function describes the conversion process from the three-dimensional point cloud and image data to the two-dimensional BEV representation. This conversion process involves complex geometric transformations and depth estimation, including the correction of camera intrinsics and extrinsics. The advantage of the BEVFusion algorithm is that it can make full use of the spatial accuracy of radar point cloud data and the texture information of video image data, and improve the accuracy and robustness of target detection through fusion in the BEV space. In the application scenario of high-altitude work safety monitoring, this means that the system can more accurately identify and locate people and objects in the working area, detect potential safety threats in a timely manner, and issue early warnings through the alarm module, thereby improving work safety.

[0072] Through the BEVFusion algorithm, this patent implements a simple and effective multimodal data fusion framework, which not only improves the performance of target detection, but also enhances the robustness of the system in the face of radar or camera failures, which is crucial to ensuring the safety of high-altitude operations.

[0073] 3. The Complex-YOLO algorithm is integrated into the high-altitude work safety space detection system to achieve real-time 3D detection of objects in the working area. The Complex-YOLO algorithm extends the YOLOv2 network and estimates multi-class 3D bounding boxes in Cartesian space through a specific complex regression strategy. The algorithm proposes a specific Euler region proposal network (E-RPN) to estimate the pose of the object, and avoids the singularity problem in single-angle estimation by adding imaginary and real parts to the regression network. The above-mentioned point cloud conversion is required: the 3D point cloud data collected by the radar monitoring module is converted into a bird's-eye view (Birds-Eye-View, BEV) RGB image. This step is inspired by MV3D and encodes the point cloud data into height, intensity and density information.

[0074] Network architecture: The Complex-YOLO network uses a simplified YOLOv2 CNN architecture, extending the complex angle regression and E-RPN to detect accurate multi-class oriented 3D objects in real time; E-RPN parses 3D position, object size, and probability and category scores for each grid cell, while estimating the orientation of the object. Through complex angles, E-RPN can be directly embedded in the network based on the imaginary and real parts to estimate the accurate object orientation. The network optimization loss function Loss is based on the concept of YOLO, and minimizes the difference between the predicted and true labels through the complex Euler regression part LEuler.

[0075] By applying the Complex-YOLO algorithm, the aerial work safety space detection system of the present invention can accurately detect and locate objects in the aerial work area, including vehicles, pedestrians, and bicycles, while monitoring the aerial work area in real time, thereby improving the safety and efficiency of the work. In addition, the system can also predict the trajectory of objects, providing more comprehensive protection for the safety of aerial work.

[0076] The overall implementation steps are as follows:

[0077] System initialization and configuration:

[0078] Radar monitoring module: Install the Livox Mid360 radar at an appropriate location to ensure that it can cover the entire aerial work area.

[0079] Video surveillance module: Install Hikvision surveillance cameras and adjust their angles and focal lengths to capture clear images of the working area.

[0080] Data processing module: Set up the edge computing platform to ensure that it can receive data from radar and cameras and perform preliminary processing.

[0081] Cloud Server: Configure the cloud server and install the necessary software and operating environment to support the operation and data processing of the Complex-YOLO algorithm.

[0082] 2. Data collection and preprocessing:

[0083] Point cloud data collection: The radar monitoring module collects 3D point cloud data of the operating area in real time.

[0084] Video data acquisition: The video monitoring module captures the video stream of the working area in real time.

[0085] Data preprocessing: The data processing module preprocesses the received point cloud data and video data, including filtering, noise reduction, format conversion, etc., to improve data quality.

[0086] 3. Bird’s Eye View (BEV) Generation:

[0087] Point cloud conversion: The preprocessed 3D point cloud data is converted into a bird’s eye view (BEV) RGB map, encoding height, intensity, and density information.

[0088] 4. Complex-YOLO network training and deployment:

[0089] Network architecture: Deploy a Complex-YOLO network based on YOLOv2, including complex angle regression and E-RPN.

[0090] Model training: Use the labeled training data set to train the Complex-YOLO model so that it can recognize and locate objects of multiple categories.

[0091] Model deployment: Deploy the trained model to the cloud server to process the data from the data processing module in real time.

[0092] 5. Real-time object detection and safety warning:

[0093] Object Detection: The Complex-YOLO model on the cloud server processes BEV images in real time, detects objects in the operating area, and estimates their 3D bounding boxes and poses.

[0094] Safety warning: When non-operating personnel or other objects are detected entering the working area, the system will issue an alarm through email and warning prompts on the web interface.

[0095] 6. System monitoring and feedback:

[0096] Monitoring interface: The video stream and radar point cloud data of the operation area, as well as the bounding boxes of detected objects, are displayed in real time through the web interface.

[0097] Alarm records: The system records all security warning events, including time, location and related image data for subsequent analysis and auditing.

[0098] (2) Equipment:

[0099] The high-altitude work safety space detection system of the present invention includes the following necessary components:

[0100] Radar monitoring module 1: uses Livox Mid360 laser radar, which is responsible for collecting three-dimensional spatial data of the working area and presenting it in the form of point cloud.

[0101] Video surveillance module 2: uses Hikvision surveillance cameras to capture video images of the working area.

[0102] Data processing module 3: responsible for data preprocessing and fusion, including radar point cloud data and video image data.

[0103] Alarm module 4: triggers an alarm when an abnormality is detected, and issues an alarm via email and warning prompts on the web interface.

[0104] Data server 5: stores and processes data, including radar point cloud data and video image data.

[0105] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0106] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0107] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A safe space detection system for high-altitude operations, characterized in that: It includes radar monitoring module, video monitoring module, data processing module, alarm module and data server; The radar monitoring module is used to collect three-dimensional spatial data of the operating area; The video monitoring module is used to collect video images; The data processing module is used for data preprocessing and fusion; The alarm module is used to trigger an alarm when an abnormality is detected; Data servers are used to store and process data.

2. The high-altitude work safety space detection system according to claim 1 is characterized in that: The radar monitoring module adopts Livox radar; The video surveillance module adopts Hikvision surveillance camera; The data processing module uses the Ubuntu operating system for processing.

3. A method for detecting a safe space for aerial work, the method being based on the detection system according to any one of claims 1 to 2, characterized in that: The method comprises the following steps: Step 1: The radar monitoring module is started to collect data; Step 2: The video surveillance module is started to collect data; Step 3: The data preprocessing module preprocesses the received data; Step 4: The data processing module independently processes the lidar point cloud and video stream. It includes two branches. One branch extracts point cloud features to predict 3D information, and the other branch extracts video image features to predict 3D information. The features of both are then projected into a unified BEV (bird's eye view) space, where they are fused and the deep learning model is used for object detection. Step 5: Assign corresponding weights to the judgment results of the video and the judgment results of the laser radar. When the judgment condition is triggered, it is considered that non-operating personnel or other objects have entered the working area, and it is determined that the construction personnel are unsafe at this time. If not, return to step 4, if yes, proceed to the next step; Step 6: The alarm module issues an alarm through emails and warning prompts on the web interface, and records the alarm event; Step 7: If you need to continue monitoring, return to step 3, otherwise end the monitoring.

4. The method for detecting a safe space for aerial work according to claim 1, characterized in that: The specific process of step 1 is as follows: start the robot operating system (ROS) on the edge computing platform, and then start the ROS driver of the Livox laser radar, which is officially provided by Livox; by configuring the corresponding ROS topic configuration file, the point cloud data and the corresponding data are published in the form of a topic, and the websocket function is used to enable the web end to receive the point cloud data of the radar; after successful startup, the Livox laser radar starts to scan the working area and collect three-dimensional spatial data.

5. The method for detecting safe space for aerial work according to claim 1, characterized in that: The specific process of step 2 is as follows: use the camera software development kit (Software Development Kit) of the video surveillance module to configure the camera's operating environment so that the program can find related dependency packages; fill in the camera's static address IP, user, password and open port, so that the real-time camera image can be obtained; the video surveillance module starts to capture video images of the working area.

6. The method for detecting a safe space for aerial work according to claim 1, characterized in that: The specific contents of the object detection algorithm used in the object detection process in step 4 are as follows: I: Image preprocessing: denoising and normalizing the images in the video stream; II: Feature Extraction: Extract key features from images using deep learning models; III: Object Recognition: Match the extracted features with the categories in the pre-trained YOLOv8 model to identify objects; IV: Confidence assessment: Conduct confidence assessment on the recognition results to determine the accuracy of the recognition; V: Threshold judgment: If the confidence is higher than the preset threshold, the detected object is considered to be a real target, otherwise it is ignored; VI: Alarm decision: Determine whether to trigger an alarm based on the detection results and safety rules.

7. The method for detecting safe space for aerial work according to claim 6, characterized in that: The image preprocessing in step 1 includes histogram equalization and Gaussian blur; The feature extraction in step II is performed using multiple layers of convolutional neural networks; The threshold in step V is dynamically adjusted according to the actual application scenario and security requirements; The alarm decision in step VI takes into account historical alarm data and real-time environmental factors.

Images