Lidar background modeling and virtual fence construction safety monitoring system and method

Abstract

The application discloses a kind of laser radar background modeling and virtual fence's construction safety monitoring system and method, it is related to construction safety monitoring technical field.The system includes laser radar point cloud acquisition module, camera acquisition module, edge computing processing terminal, three-dimensional planning client, sound-light alarm module, network communication module and power supply module.The method includes: by multi-frame statistics construction voxel background map, filters static interference;The virtual fence that user draws is converted into radar coordinate system and is screened in area;Effective change target is extracted using background neighborhood matching and clustering filtering, triggers alarm after being confirmed by continuous frame;Color point cloud visualization is generated by fusing image data.The application solves the problems of insufficient three-dimensional space monitoring capability, poor static background interference resistance, inflexible fence configuration and other problems in the prior art.

Description

Technical Field

[0001] This invention relates to the field of construction safety monitoring technology, and in particular to a construction safety monitoring system and method using lidar background modeling and virtual fences. Background Technology

[0002] Currently, common safety monitoring methods at construction sites, power supply stations, and near-electric work areas mainly include manual monitoring, video surveillance, infrared beams, ordinary electronic fences, single-point ranging, ordinary camera identification, and manual inspections after manually delineating hazardous areas. While these methods can alert personnel to dangerous areas to some extent, they still have significant shortcomings in complex construction environments, outdoor power supply equipment scenarios, and three-dimensional spatial boundary assessment.

[0003] Among these methods, manual monitoring relies on personnel experience, making it difficult to achieve continuous 24 / 7 monitoring. It also suffers from issues such as fatigue, obstructed view, delayed response, and untraceable records. Ordinary video surveillance primarily relies on two-dimensional images, lacking true three-dimensional distance and height information, making it difficult to accurately determine whether personnel, machinery, or objects have entered a specific spatial area. Furthermore, its stability decreases under conditions of backlight, nighttime, rain, fog, obstructions, complex backgrounds, or camera shake.

[0004] Infrared beam detectors, ordinary electronic fences, and single-point ranging devices can usually only form linear or simple planar boundaries, which are difficult to represent irregular polygonal areas, height limit areas, and safety boundary areas in the construction site, and cannot simultaneously determine the "horizontal range" and "height range".

[0005] Furthermore, while some existing 3D point cloud monitoring solutions can acquire spatial point clouds, they often use fixed rectangular ranges or simple distance thresholds, resulting in insufficient flexibility in area configuration. They cannot allow real-time deployment of polygonal electronic fences composed of multiple points from the client and lack a visual editing mechanism for 3D client-side linkage. Some point cloud differential solutions have not established stable background voxel models, easily misidentifying static equipment, ground, walls, fences, supports, and other background points as targets. They also lack continuous frame confirmation and clustering filtering mechanisms for point cloud noise, transient flying points, and a small number of reflection points, leading to frequent false alarms.

[0006] In summary, there is an urgent need for a safety monitoring system and method that can use LiDAR to create a 3D background, support client-side distribution of polygonal electronic fences, determine safe areas based on spatial height range, filter background changes and point cloud noise, and transmit point cloud reconstruction and alarm results in real time. Summary of the Invention

[0007] This invention provides a construction safety monitoring system and method based on lidar background modeling and virtual fencing. The invention filters out static environmental interference through voxelized background modeling, precisely defines the monitoring spatiotemporal range using a three-dimensional virtual fence, and achieves color visualization by combining camera images, thereby realizing accurate, flexible, and intuitive construction safety monitoring.

[0008] In a first aspect, embodiments of the present invention provide a construction safety monitoring system for LiDAR background modeling and virtual fence, the system comprising: a LiDAR point cloud acquisition module, a camera acquisition module, an edge computing processing terminal, a 3D planning client, an audible and visual alarm module, a network communication module, and a power supply module; The lidar point cloud acquisition module is used to acquire three-dimensional spatial point cloud data of the construction area, parse the point cloud data to obtain a raw point cloud data stream containing point coordinates and reflection intensity information, and transmit the raw point cloud data stream to the edge computing processing terminal to provide basic data for the edge computing processing terminal's background mapping, area detection, point cloud display and alarm judgment. The camera acquisition module is used to continuously acquire color image frames of the monitoring site and output real-time image data streams to the edge computing processing terminal, providing image sources for point cloud coloring and personnel recognition of the edge computing processing terminal; The 3D planning client is used to receive color point cloud data and recognition image data transmitted back from the edge computing processing terminal, so as to display the 3D point cloud and view the recognition screen; to generate detection area configuration data containing the coordinates of the region vertex and the height range in response to the user's drawing operation, and to send it to the edge computing processing terminal through the network communication module to set the detection area; and to receive and display alarm status. The edge computing processing terminal is used to receive the raw point cloud data stream, real-time image data stream and detection area configuration data, perform background modeling, coordinate transformation, area judgment, change detection and clustering filtering, output alarm control signals to the sound and light alarm module, and output color point cloud data and recognition image data back to the 3D planning client. The audible and visual alarm module is used to respond to the alarm control signal, execute audible and visual alarm actions, and trigger or deactivate the audible and visual alarm. The network communication module is used to establish a data transmission channel between the edge computing processing terminal and the 3D planning client. The power supply module is used to connect to external AC mains power and converts the AC power into the DC operating voltage required by each hardware module of the system through an internal power conversion unit.

[0009] In one optional implementation, the edge computing processing terminal includes a background voxel mapping module, a region communication and management module, a change detection and clustering module, an alarm control module, a color point cloud transmission module, and an image personnel detection module. The background voxel mapping module is used to extract static scene point clouds from the original point cloud data stream when the device is stationary, generate a background voxel map after discretization, and provide the background voxel map to the change detection and clustering module as a background reference benchmark. The regional communication and management module is used to receive the detection area configuration data sent by the 3D planning client, convert the vertex coordinates in the detection area configuration data into radar coordinate system, generate a radar coordinate system detection area snapshot, and provide the radar coordinate system detection area snapshot to the change detection and clustering module as a filtering condition. The change detection and clustering module is used to receive the real-time input raw point cloud data stream, use the radar coordinate system to detect the area snapshot to filter out the point cloud in the area, combine the background voxel map to filter out static background points to extract change points, and perform clustering and validity judgment on the change points, and output the target detection result to the alarm control module. The alarm control module is used to perform continuous frame de-jitter confirmation on the target detection result and generate the alarm control signal; The color point cloud sending module is used to sample and color the point cloud according to the radar and camera calibration parameters, and send the color point cloud in packets to the 3D planning client. The image personnel detection module is used to independently read camera images, identify personnel targets, and output the framed recognition image to the 3D planning client.

[0010] In one optional implementation, the background voxel mapping module is further used to count the number of times voxels appear in multiple background frames, and retain only voxels whose appearance ratio is not less than a preset threshold and whose corresponding positions do not have any targets to be alarmed, to form a background set in the background voxel map.

[0011] In one optional implementation, the detection area configuration data includes the area number, area activation status, minimum height, maximum height, and polygon bottom point set; The area communication and management module is further used to combine the vertices of the polygon's bottom surface with the minimum height and the maximum height to construct a detection area in the shape of a vertical cylinder, and map it to the polygonal plane range and height range in the radar coordinate system to form a snapshot of the detection area in the radar coordinate system.

[0012] Secondly, embodiments of the present invention provide a construction safety monitoring method for lidar background modeling and virtual fences. Based on the construction safety monitoring system, the method includes: The lidar point cloud acquisition module and the camera acquisition module are started. The lidar point cloud acquisition module outputs a three-dimensional point cloud, and the camera acquisition module acquires the latest image frame, generating a raw point cloud data stream and a real-time image data stream. The edge computing processing terminal loads camera intrinsic parameters, lens distortion parameters, and radar-to-camera rotation matrix and translation vector to generate calibration parameters; When the device is stationary, the background voxel mapping module of the edge computing processing terminal collects the static point cloud in the original point cloud data stream, stores the static point cloud in the field according to the voxel size, and obtains the background voxel map, which serves as the background reference benchmark for determining background points. The 3D planning client sends detection area data based on the user's drawing results, and parses the area number, activation status, minimum height, maximum height and several plane vertices of each area in the detection area data to generate detection area configuration data. The edge computing processing terminal's regional communication and management module receives the detection region configuration data, converts the region vertices in the display coordinate system to the radar coordinate system, obtains a radar coordinate system detection region snapshot, and writes it to the region manager as the basis for region determination in point cloud filtering. The change detection and clustering module of the edge computing processing terminal periodically extracts the current window point cloud from the original point cloud data stream, uses the radar coordinate system to detect the region snapshot, determines whether the Z height of each point in the region snapshot is within the region height range and whether the XY plane projection is inside the region polygon, and filters to generate a set of point clouds located within the detection region. The change detection and clustering module calls the background voxel neighborhood query function for the point cloud set located within the detection area and compares it with the background voxel map. If the current point is located within the background voxel neighborhood, it is determined to be a static background; otherwise, it is determined to be a newly added change point, thus obtaining a set of change points. When the number of points in the set of change points reaches the minimum threshold, the change detection and clustering module performs spatial clustering on the change points to obtain several change targets. Invalid targets are filtered out based on the number of cluster points, cluster size, height range, and spatial continuity to obtain the effective change target detection results. The alarm control module of the edge computing processing terminal increases the number of consecutive hit frames when a valid change target is detected, and increases the number of consecutive miss frames when no valid target is detected, based on the detection results of the valid change target. An alarm is triggered only when the number of consecutive hit frames reaches the alarm confirmation threshold, and an alarm control signal is generated to drive the audible and visual alarm module. The edge computing processing terminal's color point cloud sending module uses calibration parameters to project points in the original point cloud data stream onto the image plane of the real-time image data stream for coloring, generating color point cloud data and frame-recognition images, and then sends them to the 3D planning client via independent link packets to visualize the monitoring results.

[0013] In one alternative implementation, the region vertices in the display coordinate system are converted to the radar coordinate system. The specific process is as follows: Define the lidar coordinate system as L , L The X-axis is forward, the Y-axis is left, and the Z-axis is upward, and the display coordinate system is defined as follows: U , U The X-axis points to the right, the Y-axis points upward, and the Z-axis points forward; Define a general coordinate transformation relationship, the formula is: In the formula, To display points in the coordinate system; This is the corresponding point in the lidar coordinate system; To display the rotation matrix from the coordinate system to the radar coordinate system; To display the translation vector between the coordinate system and the origin of the lidar coordinate system; Define the rotation matrix from the display coordinate system to the lidar coordinate system as follows: In the formula, To display the rotation matrix from the coordinate system to the lidar coordinate system; The coordinate transformation formula for converting region vertices in the display coordinate system to the lidar coordinate system is defined as follows: In the formula, To display the translation vector between the coordinate system and the origin of the lidar coordinate system; This represents the X-axis coordinates of a point in the lidar coordinate system. This represents the Y-coordinate of a point in the lidar coordinate system. This represents the Z-axis coordinates of a point in the lidar coordinate system. To display the X-axis coordinates of a point in the coordinate system; To display the Y-coordinate values ​​of a point in the coordinate system; To display the Z-axis coordinates of a point in the coordinate system; The edge computing processing terminal converts the vertices of the detection area in the display coordinate system into polygons and height ranges in the radar coordinate system, forming a snapshot of the detection area in the radar coordinate system.

[0014] In one alternative implementation, the ray parity rule is used to determine whether the XY plane projection of each point in the region snapshot is inside the region polygon. The specific process is as follows: Let the adjacent vertices of the polygon be... and A ray is emitted from the point to be measured along the positive X-axis; When all the determination formulas are true, the determination ray intersects the edge once. The determination formulas are: In the formula, The coordinates of the point to be measured on the X-axis; The coordinates of the point to be measured on the Y-axis; For the polygon's th j One vertex; j For vertex index; For the first j The coordinates of each vertex on the X-axis; For the first j The Y-coordinates of each vertex; For the first j+ The X-coordinate of a vertex on the x-axis; For the first j+ The Y-coordinate of a vertex; Count the total number of valid intersections. If the total number of valid intersections is odd, the point is determined to be inside the polygon. If the point is located on the boundary of the polygon, it is treated as entering the region.

[0015] In one optional implementation, the specific process of the background voxel neighborhood query is as follows: Let the side length of the voxel be... s For any point to be measured p The formula for calculating voxel indices is: In the formula, The coordinates of the point to be measured on the X-axis; The coordinates of the point to be measured on the Y-axis; The coordinates of the point to be measured on the Y-axis; s Let the voxel side length be denoted as 'side length'. p Index the points to be measured; For voxel indexing; To represent the floor function; If there are background voxels belonging to the background set in the background voxel map. B , making ,in, b As background voxels, B For background collection, r The neighborhood radius, If it is an infinite norm, then determine the point to be measured. p If it is a background point, then it is considered a change point; The method for constructing the background voxel map is as follows: Statistical voxels in Number of times it appears in each background frame ,in, To preset the number of background frames, For the number of occurrences, only the proportion of occurrences is retained. A set of voxels with a value not lower than a preset threshold is formed to create a background set.

[0016] In one optional implementation, the specific process of change point clustering and target validity filtering is as follows: Spatial clustering of the set of change points, if the first change point Second point of change The European distance satisfies Then, an adjacency relationship is established, and the resulting connected components constitute a cluster of changing targets, where... This is the first point of change; This is the second point of change. The first point of change Second point of change European distance, The Euclidean distance threshold; Calculate the number of points in each cluster and its dimensions along the X, Y, and Z coordinate axes; The formula for the validity judgment condition is: In the formula, For the first k The number of points in each cluster; k For clustered indexes; The minimum cluster point threshold; Let X be the size of the cluster along the X-axis. Let Y be the size of the cluster along the Y-axis. Let Z be the size of the cluster along the Z-axis. This is the lower limit of the width dimension; This represents the upper limit of the width dimension; This is the lower limit of the depth dimension; This represents the upper limit of the depth dimension; This is the lower limit of the height dimension; This represents the upper limit of the height dimension; A target cluster is considered a valid change target only if it meets the above conditions.

[0017] In one optional implementation, the specific methods for split communication and visualization are as follows: The edge computing processing terminal independently transmits color point cloud data, framed recognition images, and alarm status through different ports or protocol identifiers; Data transmission adopts a packet mechanism. Each data packet contains a frame number and a packet sequence number. The client only assembles the latest frame data in the current session. If the old frame data has not been fully received and the new frame has arrived, the old frame data is discarded. The point cloud coloring process uses camera intrinsic and extrinsic parameters to project radar points onto the image plane, calculates the projection pixel coordinates, and if the projection coordinates are valid, reads the image color and assigns it to the point cloud; otherwise, it uses the radar reflection intensity grayscale value for display.

[0018] The technical solution provided in this application has at least the following beneficial effects: 1) Strong anti-interference capability and low false alarm rate: The present invention adopts a voxel-based multi-frame statistical background modeling method, which can effectively filter out fixed backgrounds such as static equipment, ground, and walls in the construction site; combined with background voxel neighborhood query and continuous frame confirmation mechanism, it can eliminate random interference such as point cloud noise, instantaneous flying points, and rain and fog reflection, which significantly improves the robustness of the monitoring system and solves the problem of easy false alarms in traditional single-frame detection.

[0019] 2) High accuracy of three-dimensional spatial monitoring: Compared with infrared beams and ordinary electronic fences, which can only form planar or linear protection, this invention constructs a vertical column detection area that includes a height range. It can simultaneously determine the horizontal position and height information of the target, realize true three-dimensional spatial protection, and meet the safety monitoring needs of complex construction scenarios (such as high-altitude operations and deep foundation pits).

[0020] 3) Flexible and intuitive area configuration: Supports real-time configuration of polygonal electronic fences from the client, allowing users to flexibly adjust the monitoring range according to the construction progress, solving the problems of fixed monitoring areas and difficult adjustments in traditional solutions. Automatic conversion between the display coordinate system and the radar coordinate system enables WYSIWYG visual editing, lowering the operational threshold.

[0021] 4) High visualization of monitoring results: By integrating lidar and visible light image data, colored 3D point clouds are generated and transmitted in real time, which makes up for the lack of texture and difficulty in identifying targets in pure radar data. It provides monitoring personnel with an intuitive and realistic 3D scene of the scene, which facilitates quick confirmation of the alarm and remote command.

[0022] 5) Good system reliability and real-time performance: The system adopts a multi-channel independent network transmission mechanism to separate the alarm detection link from the visualization link. Even if the image data transmission is blocked, the core alarm function can still operate independently, ensuring the reliability of safety monitoring. Attached Figure Description

[0023] Figure 1This is a schematic diagram of the functional modules of a construction safety monitoring system for laser radar background modeling and virtual fence provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the functional modules of the edge computing processing terminal provided in an embodiment of the present invention; Figure 3 A flowchart illustrating the steps of a construction safety monitoring method for lidar background modeling and virtual fence provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0025] The present invention will be further described below with reference to the accompanying drawings.

[0026] like Figure 1 As shown, an embodiment of the present invention provides a construction safety monitoring system 100 for LiDAR background modeling and virtual fence, the system including: LiDAR point cloud acquisition module 101, camera acquisition module 102, edge computing processing terminal 103, 3D planning client 104, sound and light alarm module 105, network communication module 106, and power supply module 107. The lidar point cloud acquisition module 101 is used to acquire three-dimensional spatial point cloud data of the construction area, parse the point cloud data to obtain an original point cloud data stream containing point coordinates and reflection intensity information, and transmit the original point cloud data stream to the edge computing processing terminal to provide basic data for the edge computing processing terminal's background mapping, area detection, point cloud display and alarm judgment. The camera acquisition module 102 is used to continuously acquire color image frames of the monitoring site and output real-time image data streams to the edge computing processing terminal, providing image sources for point cloud coloring and personnel recognition of the edge computing processing terminal. The 3D planning client 104 is used to receive color point cloud data and recognition image data transmitted back from the edge computing processing terminal, so as to display the 3D point cloud and view the recognition screen; to generate detection area configuration data containing the coordinates of the region vertex and the height range in response to the user's drawing operation, and to send it to the edge computing processing terminal through the network communication module to set the detection area; and to receive and display alarm status. The edge computing processing terminal 103 is used to receive the original point cloud data stream, real-time image data stream and detection area configuration data, perform background modeling, coordinate transformation, area judgment, change detection and clustering filtering, output alarm control signals to the sound and light alarm module, and output color point cloud data and recognition image data back to the three-dimensional planning client. The audible and visual alarm module 105 is used to respond to the alarm control signal, perform audible and visual alarm actions, and trigger or deactivate the audible and visual alarm. The network communication module 106 is used to establish a data transmission channel between the edge computing processing terminal and the 3D planning client. The power supply module 107 is used to connect to external AC mains power and convert AC power into DC operating voltage required by each hardware module of the system through an internal power conversion unit.

[0027] In one alternative implementation, such as Figure 2 As shown, the edge computing processing terminal 103 includes a background voxel mapping module 1031, a regional communication and management module 1032, a change detection and clustering module 1033, an alarm control module 1034, a color point cloud sending module 1035, and an image personnel detection module 1036. The background voxel mapping module 1031 is used to extract static scene point clouds from the original point cloud data stream when the device is stationary, generate a background voxel map after discretization, and provide the background voxel map to the change detection and clustering module as a background reference benchmark. The regional communication and management module 1032 is used to receive the detection area configuration data sent by the 3D planning client, convert the vertex coordinates in the detection area configuration data into a radar coordinate system, generate a radar coordinate system detection area snapshot, and provide the radar coordinate system detection area snapshot to the change detection and clustering module as a filtering condition. The change detection and clustering module 1033 is used to receive the real-time input raw point cloud data stream, use the radar coordinate system to detect the area snapshot to filter out the point cloud in the area, combine the background voxel map to filter out static background points to extract change points, and perform clustering and validity judgment on the change points, and output the target detection result to the alarm control module. The alarm control module 1034 is used to perform continuous frame de-jitter confirmation on the target detection result and generate the alarm control signal. The color point cloud sending module 1035 is used to sample and color the point cloud according to the radar and camera calibration parameters, and send the color point cloud in packets to the 3D planning client. The image personnel detection module 1036 is used to independently read camera images, identify personnel targets, and output the framed recognition image to the 3D planning client.

[0028] In one optional implementation, the background voxel mapping module is further used to count the number of times voxels appear in multiple background frames, and retain only voxels whose appearance ratio is not less than a preset threshold and whose corresponding positions do not have any targets to be alarmed, to form a background set in the background voxel map.

[0029] In one optional implementation, the detection area configuration data includes the area number, area activation status, minimum height, maximum height, and polygon bottom point set; Among them, the area number is used to distinguish different electronic fences or alarm areas; the area activation status is used to control whether the area participates in detection, when it is 0, the area does not participate in detection, and when it is 1, it participates in detection; the minimum height and maximum height correspond to the height values ​​in the coordinate system displayed by the 3D planning client, and after coordinate transformation, they correspond to the Z-axis height range in the LiDAR coordinate system; the polygon bottom point set is a sequence of XZ plane points in the display coordinate system arranged clockwise or counterclockwise. The process of determining whether a point is located within the detection area includes: first, determining whether the area is enabled; second, determining whether the area polygon contains at least three points; third, determining whether the height of the point falls between the minimum height (minHeight) and the maximum height (maxHeight); and finally, using the ray method to determine whether the horizontal projection of the point falls inside the polygon. If the point falls exactly on the boundary of the polygon, it is treated as having entered the area. The area communication and management module is further used to combine the vertices of the polygon's bottom surface with the minimum height and the maximum height to construct a detection area in the shape of a vertical cylinder, and map it to the polygonal plane range and height range in the radar coordinate system to form a snapshot of the detection area in the radar coordinate system.

[0030] In one optional implementation, the operating strategy of the image person detection module is as follows: This module operates independently of the safety alarm link and is mainly used to provide operators with a reference of the person target in the camera screen. This module reads the latest image frame from the camera manager, does not build a continuously growing image queue, and automatically skips the old frame and processes the current latest frame when the inference speed is lower than the camera frame rate. The recognition result is sent to the client as an image with a person frame. This design allows image inference and lidar detection to operate independently, so that even if image personnel detection fails or the image communication link is interrupted, radar background subtraction, area detection, and audible and visual alarms can still operate independently.

[0031] In one optional implementation, the alarm control module adopts a continuous detection window confirmation strategy. Each detection window outputs whether a valid changing target exists. If multiple consecutive windows detect a valid changing target, the system triggers an alarm. If multiple consecutive windows fail to detect a valid target after the alarm, the system deactivates the alarm. This strategy is used to filter out instantaneous noise points, a small number of flying points, raindrops, dust, and short-term reflection anomalies, thereby reducing false alarms and frequent switching of alarm states.

[0032] In one optional implementation, the specific communication configuration of the network communication module is as follows: Different types of data are transmitted using different ports or different protocol identifiers to avoid data blocking. The region configuration data is sent from the 3D planning client to the edge computing processing terminal via UDP port 9200, with the protocol identifier "RGON". After receiving the data, the edge terminal converts it into radar coordinates and updates the detection region manager. Colored point cloud data is sent from the edge computing processing terminal to the 3D planning client via UDP port 9000. The protocol is identified as "PCLD". The data includes frame number, packet sequence number, total number of points, point step size and timestamp. The client reassembles the latest complete frame according to the frame number and displays it through ComputeBuffer. The frame-recognition image is sent from the edge computing processing terminal to the 3D planning client via UDP port 9100. The protocol identifier is "YIMG". The data includes frame number, packet sequence number, image size and number of people. The client reassembles the latest image frame and loads it as a Texture2D display. Alarm control signals are output to the audible and visual alarm devices via I / O interfaces, relays, or network interfaces.

[0033] Secondly, embodiments of the present invention provide a construction safety monitoring method for lidar background modeling and virtual fences, such as... Figure 3 As shown, based on the construction safety monitoring system, the method includes: S201: Start the lidar point cloud acquisition module and the camera acquisition module. The lidar point cloud acquisition module outputs a three-dimensional point cloud, and the camera acquisition module acquires the latest image frame, generating a raw point cloud data stream and a real-time image data stream. S202: The edge computing processing terminal loads camera intrinsic parameters, lens distortion parameters, and radar-to-camera rotation matrix and translation vector to generate calibration parameters; S203: When the device is stationary, the background voxel mapping module of the edge computing processing terminal collects the static point cloud in the original point cloud data stream, stores the static point cloud in the field according to the voxel size, and obtains the background voxel map, which serves as the background reference benchmark for determining the background points. In this step, the method for constructing the background voxel map is as follows: Let the side length of the voxel be... s For any point to be measured p The formula for calculating voxel indices is: In the formula, The coordinates of the point to be measured on the X-axis; The coordinates of the point to be measured on the Y-axis; The coordinates of the point to be measured on the Y-axis; s Let the voxel side length be denoted as 'side length'. p Index the points to be measured; For voxel indexing; To represent the floor function; Statistical voxels in Number of times it appears in each background frame ,in, To preset the number of background frames, The number of times it appears. For voxel indexing, only the occurrence ratio is retained. A set of voxels with a value not lower than a preset threshold is formed; In the formula, To determine the proportion threshold; For background frame index; S204: The 3D planning client sends detection area data based on the user's drawing results, parses the area number, activation status, minimum height, maximum height and several plane vertices of each area in the detection area data, and generates detection area configuration data. S205: The regional communication and management module of the edge computing processing terminal receives the detection area configuration data, converts the regional vertices in the display coordinate system to the radar coordinate system, obtains a radar coordinate system detection area snapshot, and writes it to the regional manager as the basis for determining the region for point cloud filtering. In this step, the region vertices in the display coordinate system will be converted to the radar coordinate system. The specific process is as follows: Define the lidar coordinate system as L , L The X-axis is forward, the Y-axis is left, and the Z-axis is upward, and the display coordinate system is defined as follows: U , U The X-axis points to the right, the Y-axis points upward, and the Z-axis points forward; Define a general coordinate transformation relationship, the formula is: In the formula, To display points in the coordinate system; A point in the lidar coordinate system; To display the rotation matrix from the coordinate system to the radar coordinate system; To display the translation vector between the coordinate system and the origin of the lidar coordinate system; Define the rotation matrix from the display coordinate system to the lidar coordinate system as follows: In the formula, To display the rotation matrix from the coordinate system to the lidar coordinate system; The coordinate transformation formula for converting region vertices in the display coordinate system to the lidar coordinate system is defined as follows: In the formula, To display the translation vector between the coordinate system and the origin of the lidar coordinate system; This represents the X-axis coordinates of a point in the lidar coordinate system. This represents the Y-coordinate of a point in the lidar coordinate system. This represents the Z-axis coordinates of a point in the lidar coordinate system. To display the X-axis coordinates of a point in the coordinate system; To display the Y-coordinate values ​​of a point in the coordinate system; To display the Z-axis coordinates of a point in the coordinate system; It is the transpose symbol; The edge computing processing terminal converts the vertex of the detection area in the display coordinate system into a polygon and height range in the radar coordinate system, forming a snapshot of the detection area in the radar coordinate system. S206: The change detection and clustering module of the edge computing processing terminal periodically extracts the current window point cloud from the original point cloud data stream, uses the radar coordinate system to detect the region snapshot, determines whether the Z height of each point in the region snapshot is within the region height range and whether the XY plane projection is inside the region polygon, and filters to generate a set of point clouds located within the detection region. In this step, let the first... i Each detection area is Its horizontal base is The height range is [ , ],in, i For detection area indexing, For the first i Each detection area For the first i The horizontal bottom surface of each detection area For the first i The lower limit of the height of each detection area. For the first i The upper limit of the height of each detection area; Any point to be measured p The criteria for determining whether a region belongs to this area are: In the formula, p Index the points to be measured; For the first i The activation status of each detection area =1 indicates that the area is in an "on" state and is participating in monitoring. If the value is 0, the region is ignored and not detected. It is a logically equivalent symbol (if and only if); The logical AND operator indicates that all conditions for connection must be met simultaneously. The ray parity rule is used to determine whether the XY plane projection of each point in the region snapshot is inside the region polygon. The specific process is as follows: Let the adjacent vertices of the polygon be... and A ray is emitted from the point to be measured along the positive X-axis; When all the determination formulas are true, the determination ray intersects the edge once. The determination formulas are: In the formula, The coordinates of the point to be measured on the X-axis; The coordinates of the point to be measured on the Y-axis; For the polygon's th j One vertex; j For vertex index; For the first j The coordinates of each vertex on the X-axis; For the first j The Y-coordinates of each vertex; For the first j+ The X-coordinate of a vertex on the x-axis; For the first j+ The Y-coordinate of a vertex; Count the total number of valid intersection points. If the total number of valid intersection points is odd, the point is determined to be inside the polygon. If the point is located on the polygon boundary, it is treated as entering the region. The formula is: In the formula, This is a simplified calculation result of the cross product; A line segment is defined as an edge of a polygon, defined by the two endpoints of its edge. a Starting from, The endpoint; For vectors The Euclidean norm (length) represents the line segment. Length; Tolerance factor for boundary judgment; For line segments ; p Index the points to be measured; S207: The change detection and clustering module calls the background voxel neighborhood query function for the point cloud set located within the detection area, and compares it with the background voxel map. If the current point is located within the background voxel neighborhood, it is determined to be a static background; otherwise, it is determined to be a newly added change point, and a set of change points is obtained. In this step, the specific process of background voxel neighborhood query is as follows: If there are background voxels belonging to the background set in the background voxel map. B , making ,in, This is a background detection function; a return value of 1 indicates that the point belongs to the background, and a return value of 0 indicates that the point belongs to the change point. b As background voxels, B For background collection, r The neighborhood radius, If the norm is infinite (Chebyshev distance), then the point to be measured is determined. p If it is a background point, then it is considered a change point; For the point cloud of the current detection window, the formula for extracting the set of changed points is: In the formula, The set of points of change; Point cloud for the current detection window; For the first i The activation status of each detection area; i For detection area indexing; p Index the points to be measured; when When a node is identified as an isolated noise point, subsequent clustering is not performed. The minimum number of change points threshold; S208: When the number of points in the set of change points reaches the minimum threshold, the change detection and clustering module performs spatial clustering on the change points to obtain several change targets. Invalid targets are filtered out based on the number of cluster points, cluster size, height range, and spatial continuity to obtain the effective change target detection results. In this step, the specific process of change point clustering and target effectiveness filtering is as follows: Spatial clustering of the set of change points, if the first change point Second point of change The European distance satisfies Then, an adjacency relationship is established, and the resulting connected components constitute a cluster of changing targets, where... This is the first point of change; This is the second point of change. The first point of change Second point of change European distance, The Euclidean distance threshold; Calculate the number of points in each cluster and its dimensions along the X, Y, and Z coordinate axes; In the formula, For the first k The number of points in each cluster; k For clustered indexes; For changing target clusters; p Index the points to be measured; This is a function to find the maximum value. The function is for finding the minimum value; The coordinates of the point to be measured on the X-axis; The coordinates of the point to be measured on the Y-axis; z The coordinates of the point to be measured on the Z-axis; The formula for the validity judgment condition is: In the formula, For the first k The number of points in each cluster; k For clustered indexes; The minimum cluster point threshold; Let X be the size of the cluster along the X-axis. Let Y be the size of the cluster along the Y-axis. Let Z be the size of the cluster along the Z-axis. This is the lower limit of the width dimension; This represents the upper limit of the width dimension; This is the lower limit of the depth dimension; This represents the upper limit of the depth dimension; This is the lower limit of the height dimension; This represents the upper limit of the height dimension; A target cluster is considered a valid changed target only if it meets the above conditions; S209: The alarm control module of the edge computing processing terminal, based on the effective change target detection results, increases the number of consecutive hit frames when an effective change target is detected, and increases the number of consecutive miss frames when no effective target is detected. An alarm is triggered only when the number of consecutive hit frames reaches the alarm confirmation threshold, generating an alarm control signal to drive the audible and visual alarm module. S2010: The color point cloud sending module of the edge computing processing terminal uses calibration parameters to project the points in the original point cloud data stream onto the image plane of the real-time image data stream for coloring, generating color point cloud data and frame recognition images, and sending them to the 3D planning client through independent link packets to realize the visualization of monitoring results; In this step, to achieve point cloud coloring, radar points need to be projected onto the image plane. The conversion formula is as follows: In the formula, A point in the lidar coordinate system; A point in the camera coordinate system; This is the rotation matrix from the camera coordinate system to the radar coordinate system; This is the translation vector between the camera coordinate system and the origin of the lidar coordinate system; This represents the X-axis coordinates of a point in the camera coordinate system. This represents the Y-coordinate of a point in the camera coordinate system. This represents the Z-axis coordinates of a point in the camera coordinate system. It is the transpose symbol; Let the camera intrinsic parameters be... The projected pixel coordinates are calculated as follows: In the formula, The x-coordinate in the projected pixel coordinates; v The ordinate in the projected pixel coordinates; The normalized focal length is the camera's intrinsic parameter. The coordinates of the principal point for the camera's intrinsic parameters; Only when >0 and 0≤ < W ,0≤ v < H When the condition is not met, the color of the corresponding location in the image is read and assigned to the point cloud. If the condition is not met, the grayscale value of the radar reflection intensity is used for display. W The width of the image. H The height of the image; The specific methods for split communication and visualization are as follows: The edge computing processing terminal independently transmits color point cloud data, framed recognition images, and alarm status through different ports or protocol identifiers; Data transmission adopts a packet mechanism. Each data packet contains a frame number and a packet sequence number. The client only assembles the latest frame data in the current session. If the old frame data has not been fully received and the new frame has arrived, the old frame data is discarded. The point cloud coloring process uses camera intrinsic and extrinsic parameters to project radar points onto the image plane, calculates the coordinates of the projected pixels, and if the projection coordinates are valid, reads the image color and assigns it to the point cloud; otherwise, it uses the grayscale value of radar reflection intensity for display. When reading the color at a corresponding position in the image, the nearest neighbor interpolation method is used to directly read the corresponding coordinate pixel value, or the weighted average of the surrounding neighboring pixels is used to read the color to obtain a smoother coloring effect.

[0034] In one alternative implementation, the 3D planning client provides two display modes: Real-time monitoring mode: Only the latest received and reconstructed single frame point cloud data is displayed, and historical frames are not retained to ensure the real-time nature of monitoring; Scene observation mode: Enable the historical point cloud accumulation function. The client will continuously write the received point cloud data into the GPU circular buffer. When the buffer is full, it will overwrite the oldest data to enable historical backtracking and viewing of the construction scene.

[0035] The technical solution provided in this application has at least the following beneficial effects: 1) Strong anti-interference capability and low false alarm rate: The present invention adopts a voxel-based multi-frame statistical background modeling method, which can effectively filter out fixed backgrounds such as static equipment, ground, and walls in the construction site; combined with background voxel neighborhood query and continuous frame confirmation mechanism, it can eliminate random interference such as point cloud noise, instantaneous flying points, and rain and fog reflection, which significantly improves the robustness of the monitoring system and solves the problem of easy false alarms in traditional single-frame detection.

[0036] 2) High accuracy of three-dimensional spatial monitoring: Compared with infrared beams and ordinary electronic fences, which can only form planar or linear protection, this invention constructs a vertical column detection area that includes a height range. It can simultaneously determine the horizontal position and height information of the target, realize true three-dimensional spatial protection, and meet the safety monitoring needs of complex construction scenarios (such as high-altitude operations and deep foundation pits).

[0037] 3) Flexible and intuitive area configuration: Supports real-time configuration of polygonal electronic fences from the client, allowing users to flexibly adjust the monitoring range according to the construction progress, solving the problems of fixed monitoring areas and difficult adjustments in traditional solutions. Automatic conversion between the display coordinate system and the radar coordinate system enables WYSIWYG visual editing, lowering the operational threshold.

[0038] 4) High visualization of monitoring results: By integrating lidar and visible light image data, colored 3D point clouds are generated and transmitted in real time, which makes up for the lack of texture and difficulty in identifying targets in pure radar data. It provides monitoring personnel with an intuitive and realistic 3D scene of the scene, which facilitates quick confirmation of the alarm and remote command.

[0039] 5) Good system reliability and real-time performance: The system adopts a multi-channel independent network transmission mechanism to separate the alarm detection link from the visualization link. Even if the image data transmission is blocked, the core alarm function can still operate independently, ensuring the reliability of safety monitoring.

[0040] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A construction safety monitoring system for laser radar background modeling and virtual fences, characterized in that, The system includes: a lidar point cloud acquisition module, a camera acquisition module, an edge computing processing terminal, a 3D planning client, an audible and visual alarm module, a network communication module, and a power supply module; The lidar point cloud acquisition module is used to acquire three-dimensional spatial point cloud data of the construction area, parse the point cloud data to obtain a raw point cloud data stream containing point coordinates and reflection intensity information, and transmit the raw point cloud data stream to the edge computing processing terminal to provide basic data for the edge computing processing terminal's background mapping, area detection, point cloud display and alarm judgment. The camera acquisition module is used to continuously acquire color image frames of the monitoring site and output real-time image data streams to the edge computing processing terminal, providing image sources for point cloud coloring and personnel recognition of the edge computing processing terminal; The 3D planning client is used to receive color point cloud data and recognition image data transmitted back from the edge computing processing terminal, so as to display the 3D point cloud and view the recognition screen; to generate detection area configuration data containing the coordinates of the region vertex and the height range in response to the user's drawing operation, and to send it to the edge computing processing terminal through the network communication module to set the detection area; and to receive and display alarm status. The edge computing processing terminal is used to receive the raw point cloud data stream, real-time image data stream and detection area configuration data, perform background modeling, coordinate transformation, area judgment, change detection and clustering filtering, output alarm control signals to the sound and light alarm module, and output color point cloud data and recognition image data back to the 3D planning client. The audible and visual alarm module is used to respond to the alarm control signal, execute audible and visual alarm actions, and trigger or deactivate the audible and visual alarm. The network communication module is used to establish a data transmission channel between the edge computing processing terminal and the 3D planning client. The power supply module is used to connect to external AC mains power and converts the AC power into the DC operating voltage required by each hardware module of the system through an internal power conversion unit.

2. The construction safety monitoring system for lidar background modeling and virtual fence as described in claim 1, characterized in that, The edge computing processing terminal includes a background voxel mapping module, a regional communication and management module, a change detection and clustering module, an alarm control module, a color point cloud transmission module, and an image personnel detection module; The background voxel mapping module is used to extract static scene point clouds from the original point cloud data stream when the device is stationary, generate a background voxel map after discretization, and provide the background voxel map to the change detection and clustering module as a background reference benchmark. The regional communication and management module is used to receive the detection area configuration data sent by the 3D planning client, convert the vertex coordinates in the detection area configuration data into radar coordinate system, generate a radar coordinate system detection area snapshot, and provide the radar coordinate system detection area snapshot to the change detection and clustering module as a filtering condition. The change detection and clustering module is used to receive the real-time input raw point cloud data stream, use the radar coordinate system to detect the area snapshot to filter out the point cloud in the area, combine the background voxel map to filter out static background points to extract change points, and perform clustering and validity judgment on the change points, and output the target detection result to the alarm control module. The alarm control module is used to perform continuous frame de-jitter confirmation on the target detection result and generate the alarm control signal; The color point cloud sending module is used to sample and color the point cloud according to the radar and camera calibration parameters, and send the color point cloud in packets to the 3D planning client. The image personnel detection module is used to independently read camera images, identify personnel targets, and output the framed recognition image to the 3D planning client.

3. The construction safety monitoring system for lidar background modeling and virtual fence as described in claim 2, characterized in that, The background voxel mapping module is further used to count the number of times voxels appear in multiple background frames, and retain only voxels whose appearance ratio is not less than a preset threshold and whose corresponding positions do not have any targets to be alarmed, to form the background set in the background voxel map.

4. The construction safety monitoring system for lidar background modeling and virtual fence as described in claim 3, characterized in that, The detection area configuration data includes the area number, area activation status, minimum height, maximum height, and polygon bottom point set; The area communication and management module is further used to combine the vertices of the polygon's bottom surface with the minimum height and the maximum height to construct a detection area in the shape of a vertical cylinder, and map it to the polygonal plane range and height range in the radar coordinate system to form a snapshot of the detection area in the radar coordinate system.

5. A construction safety monitoring method based on lidar background modeling and virtual fence, wherein the method is based on the construction safety monitoring system as described in any one of claims 1-4, characterized in that... The method includes: The lidar point cloud acquisition module and the camera acquisition module are started. The lidar point cloud acquisition module outputs a three-dimensional point cloud, and the camera acquisition module acquires the latest image frame, generating a raw point cloud data stream and a real-time image data stream. The edge computing processing terminal loads camera intrinsic parameters, lens distortion parameters, and radar-to-camera rotation matrix and translation vector to generate calibration parameters; When the device is stationary, the background voxel mapping module of the edge computing processing terminal collects the static point cloud in the original point cloud data stream, stores the static point cloud in the field according to the voxel size, and obtains the background voxel map, which serves as the background reference benchmark for determining background points. The 3D planning client sends detection area data based on the user's drawing results, and parses the area number, activation status, minimum height, maximum height and several plane vertices of each area in the detection area data to generate detection area configuration data. The edge computing processing terminal's regional communication and management module receives the detection region configuration data, converts the region vertices in the display coordinate system to the radar coordinate system, obtains a radar coordinate system detection region snapshot, and writes it to the region manager as the basis for region determination in point cloud filtering. The change detection and clustering module of the edge computing processing terminal periodically extracts the current window point cloud from the original point cloud data stream, uses the radar coordinate system to detect the region snapshot, determines whether the Z height of each point in the region snapshot is within the region height range and whether the XY plane projection is inside the region polygon, and filters to generate a set of point clouds located within the detection region. The change detection and clustering module calls the background voxel neighborhood query function for the point cloud set located within the detection area and compares it with the background voxel map. If the current point is located within the background voxel neighborhood, it is determined to be a static background; otherwise, it is determined to be a newly added change point, thus obtaining a set of change points. When the number of points in the set of change points reaches the minimum threshold, the change detection and clustering module performs spatial clustering on the change points to obtain several change targets. Invalid targets are filtered out based on the number of cluster points, cluster size, height range, and spatial continuity to obtain the effective change target detection results. The alarm control module of the edge computing processing terminal increases the number of consecutive hit frames when a valid change target is detected, and increases the number of consecutive miss frames when no valid target is detected, based on the detection results of the valid change target. An alarm is triggered only when the number of consecutive hit frames reaches the alarm confirmation threshold, and an alarm control signal is generated to drive the audible and visual alarm module. The edge computing processing terminal's color point cloud sending module uses calibration parameters to project points in the original point cloud data stream onto the image plane of the real-time image data stream for coloring, generating color point cloud data and frame-recognition images, and then sends them to the 3D planning client via independent link packets to visualize the monitoring results.

6. The construction safety monitoring method for lidar background modeling and virtual fence according to claim 5, characterized in that, The process of converting the region vertices in the display coordinate system to the radar coordinate system is as follows: Define the lidar coordinate system as L , L The X-axis is forward, the Y-axis is left, and the Z-axis is upward, and the display coordinate system is defined as follows: U , U The X-axis points to the right, the Y-axis points upward, and the Z-axis points forward; Define a general coordinate transformation relationship, the formula is: In the formula, To display points in the coordinate system; This is the corresponding point in the lidar coordinate system; To display the rotation matrix from the coordinate system to the radar coordinate system; To display the translation vector between the coordinate system and the origin of the lidar coordinate system; Define the rotation matrix from the display coordinate system to the lidar coordinate system as follows: In the formula, To display the rotation matrix from the coordinate system to the lidar coordinate system; The coordinate transformation formula for converting region vertices in the display coordinate system to the lidar coordinate system is defined as follows: In the formula, To display the translation vector between the coordinate system and the origin of the lidar coordinate system; This represents the X-axis coordinates of a point in the lidar coordinate system. This represents the Y-coordinate of a point in the lidar coordinate system. This represents the Z-axis coordinates of a point in the lidar coordinate system. To display the X-axis coordinates of a point in the coordinate system; To display the Y-coordinate values ​​of a point in the coordinate system; To display the Z-axis coordinates of a point in the coordinate system; The edge computing processing terminal converts the vertices of the detection area in the display coordinate system into polygons and height ranges in the radar coordinate system, forming a snapshot of the detection area in the radar coordinate system.

7. The construction safety monitoring method for lidar background modeling and virtual fence according to claim 6, characterized in that, The ray parity rule is used to determine whether the XY plane projection of each point in the region snapshot is inside the region polygon. The specific process is as follows: Let the adjacent vertices of the polygon be... and A ray is emitted from the point to be measured along the positive X-axis; When all the determination formulas are true, the determination ray intersects the edge once. The determination formulas are: In the formula, The coordinates of the point to be measured on the X-axis; The coordinates of the point to be measured on the Y-axis; For the polygon's th j One vertex; j For vertex index; For the first j The coordinates of each vertex on the X-axis; For the first j The Y-coordinates of each vertex; For the first j+ The X-coordinate of a vertex on the x-axis; For the first j+ The Y-coordinate of a vertex; Count the total number of valid intersections. If the total number of valid intersections is odd, the point is determined to be inside the polygon. If the point is located on the boundary of the polygon, it is treated as entering the region.

8. The construction safety monitoring method for lidar background modeling and virtual fence according to claim 7, characterized in that, The specific process of the background voxel neighborhood query is as follows: Let the side length of the voxel be... s For any point to be measured p The formula for calculating voxel indices is: In the formula, The coordinates of the point to be measured on the X-axis; The coordinates of the point to be measured on the Y-axis; The coordinates of the point to be measured on the Y-axis; s Let the voxel side length be denoted as 'side length'. p Index the points to be measured; For voxel indexing; To represent the floor function; If there are background voxels belonging to the background set in the background voxel map. B , making ,in, b As background voxels, B For background collection, r The neighborhood radius, If it is an infinite norm, then determine the point to be measured. p If it is a background point, then it is considered a change point; The method for constructing the background voxel map is as follows: Statistical voxels in Number of times it appears in each background frame ,in, To preset the number of background frames, For the number of occurrences, only the proportion of occurrences is retained. A set of voxels with a value not lower than a preset threshold is formed to create a background set.

9. The construction safety monitoring method for lidar background modeling and virtual fence according to claim 8, characterized in that, The specific process of change point clustering and target effectiveness filtering is as follows: Spatial clustering of the set of change points, if the first change point Second point of change The European distance satisfies Then, an adjacency relationship is established, and the resulting connected components constitute a cluster of changing targets, where... This is the first point of change; This is the second point of change. The first point of change Second point of change European distance, The Euclidean distance threshold; Calculate the number of points in each cluster and its dimensions along the X, Y, and Z coordinate axes; The formula for the validity judgment condition is: In the formula, For the first k The number of points in each cluster; k For clustered indexes; The minimum cluster point threshold; Let X be the size of the cluster along the X-axis. Let Y be the size of the cluster along the Y-axis. Let Z be the size of the cluster along the Z-axis. This is the lower limit of the width dimension; This represents the upper limit of the width dimension; This is the lower limit of the depth dimension; This represents the upper limit of the depth dimension; This is the lower limit of the height dimension; This represents the upper limit of the height dimension; A target cluster is considered a valid change target only if it meets the above conditions.

10. The construction safety monitoring method for lidar background modeling and virtual fence according to claim 9, characterized in that, The specific methods for split communication and visualization are as follows: The edge computing processing terminal independently transmits color point cloud data, framed recognition images, and alarm status through different ports or protocol identifiers; Data transmission adopts a packet mechanism. Each data packet contains a frame number and a packet sequence number. The client only assembles the latest frame data in the current session. If the old frame data has not been fully received and the new frame has arrived, the old frame data is discarded. The point cloud coloring process uses camera intrinsic and extrinsic parameters to project radar points onto the image plane, calculates the projection pixel coordinates, and if the projection coordinates are valid, reads the image color and assigns it to the point cloud; otherwise, it uses the radar reflection intensity grayscale value for display.

Images