A three-dimensional point cloud intrusion detection method and device based on deep learning

Abstract

This invention discloses a method and apparatus for intrusion detection based on deep learning-based 3D point cloud. The method acquires a point cloud of the scene to be tested; it then performs target recognition on the point cloud using a personnel target detection network to obtain personnel target detection information. The personnel target detection network is constructed based on the Voxel R-CNN framework, and the feature set of the voxel grid is generated based on the position information of the points in the point cloud contained within the voxel grid and the point cloud features. Based on the detection information and the position information of dangerous areas in the scene to be tested, it determines whether an intrusion event has occurred. This invention compensates for the shortcomings of MeanVFE in feature utilization by reconstructing the voxel grid feature set generation method through the personnel target detection network, calculating the offset, and concatenating it with other features. It also achieves the aggregation of local and global information, utilizing feature information at different levels to improve feature expression quality, thus significantly improving detection accuracy.

Description

Technical Field

[0001] This invention belongs to the field of lidar area intrusion detection technology, and particularly relates to a method and device for intrusion detection based on deep learning three-dimensional point cloud. Background Technology

[0002] Coal storage yards are the core areas for coal storage, transfer, and management. Their development faces numerous challenges, the most prominent being personnel safety in hazardous working environments. Coal storage yards have a large number of machines and vehicles operating day and night, while coal piles are also at risk of landslides and collapses, making personnel safety accidents highly likely.

[0003] To ensure the safety of coal storage yards, safety barriers and warning signs are usually installed to alert unauthorized personnel. However, some individuals still violate regulations and enter the coal storage area. With the development of machine vision technology, the use of machines to replace manual inspection is becoming increasingly widespread, such as vision camera-based inspection and lidar-based inspection methods.

[0004] However, due to the complex background of the coal loading site, traditional detection methods often lack robustness and are easily interfered with by objects such as coal transport platforms and trucks, leading to misjudgments and missed judgments. Summary of the Invention

[0005] The purpose of this invention is to provide a method and apparatus for intrusion detection based on deep learning three-dimensional point cloud, so as to improve the accuracy of intrusion detection at the loading site of coal storage yard.

[0006] This invention adopts the following technical solution: a deep learning-based 3D point cloud intrusion detection method, comprising the following steps:

[0007] Obtain the point cloud of the scene to be tested;

[0008] The point cloud is used to identify targets based on the personnel target detection network to obtain the detection information of personnel targets. The personnel target detection network is built based on the Voxel R-CNN framework, and the feature set of the voxel grid is generated according to the position information of the points in the point cloud contained in the voxel grid and the point cloud features.

[0009] The determination of whether an intrusion event has occurred is based on the detection information and the location information of dangerous areas in the scene to be tested.

[0010] Furthermore, the feature set of the voxel lattice consists of enhanced features of its points, and the enhanced feature generation method is as follows:

[0011] Calculate the average position of all points in the voxel lattice, and take the point corresponding to the average position as the centroid of the voxel lattice;

[0012] Calculate the centroid offset between the point and its corresponding centroid;

[0013] Calculate the center offset of the point from the corresponding center; where the center is the center point of the voxel lattice;

[0014] The point cloud features, center offset, and centroid offset of a point are concatenated to obtain the enhanced features.

[0015] Furthermore, it also includes feature aggregation on the feature set of the voxel lattice, specifically using the following method:

[0016] The feature set is passed through three PFN layers to obtain the voxel features of the voxel lattice; in the first and second PFN layers, the enhanced features are concatenated with the pooled enhanced features.

[0017] Furthermore, the 3D backbone network in the personnel target detection network includes an input convolutional layer, four convolutional blocks, and an output convolutional layer;

[0018] The first three convolutional blocks are followed by Focal Conv blocks.

[0019] Furthermore, the TeBEVPool module is used to achieve high compression in the personnel target detection network.

[0020] Furthermore, determining whether an intrusion event has occurred based on detection information and the location information of dangerous areas in the scene under test includes:

[0021] Calculate the coordinates of the center point of the detection information;

[0022] Determine the closed polygon based on the location information of the danger zone;

[0023] Whether an intrusion has occurred is determined based on the coordinates of the center point and the closed polygon.

[0024] Furthermore, determining whether an intrusion has occurred based on the center point coordinates and the closed polygon includes:

[0025] χ=sum[q(P 2D ,n)%2],

[0026] Where χ represents the number of intersections between the center point coordinates and the closed polygon; an odd χ indicates an intrusion, and an even χ indicates no intrusion. q represents the distance from the center point coordinates P. 2D The number of actual intersections between n rays emitted in any direction and the sides of a rectangle, where %2 represents the remainder after dividing by 2, and sum represents the summation.

[0027] Furthermore, when χ is even, the shortest distance between the center point coordinates and the closed polygon is calculated, and the intrusion level is determined based on the shortest distance and the distance threshold.

[0028] Another technical solution of the present invention: a deep learning-based 3D point cloud intrusion detection device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement any of the above methods.

[0029] The beneficial effects of this invention are: by generating a feature set of a voxel grid reconstructed by a personnel target detection network, and by calculating the offset and concatenating it with other features, this invention makes up for the shortcomings of MeanVFE in feature utilization, while realizing the aggregation of local and global information, and using feature information at different levels to improve the quality of feature expression, which can significantly improve detection accuracy. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the technical route of the method of the present invention;

[0031] Figure 2 This is a schematic diagram of the overall network structure of EFT-RCNN in an embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram of the Enhanced VFE network structure in an embodiment of the present invention;

[0033] Figure 4 This is a schematic diagram of the structure of a 3D backbone network based on focal convolution in an embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram of the TeBEV Pooling structure in an embodiment of the present invention;

[0035] Figure 6 This is a schematic diagram illustrating the intrusion scenario determination in an embodiment of the present invention;

[0036] Figure 7 This is a schematic diagram showing the distance between the center point coordinates and the sides of the closed polygon under different conditions in embodiments of the present invention. Detailed Implementation

[0037] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0038] Amidst global energy market fluctuations and extreme weather events, coal reserves can rapidly release production capacity, increase coal supply, meet market demand, effectively respond to various emergencies, and ensure a stable and secure national energy supply.

[0039] Detecting intrusions into dangerous areas is a priority for enhancing safety. Various industries already have numerous measures in place to detect workers entering dangerous areas. These safety measures primarily rely on traditional management methods such as safety training and inspections. This method is time-consuming, labor-intensive, and requires comprehensive management throughout the entire process, and depends heavily on worker self-discipline and supervisory checks.

[0040] With the development of machine vision technology, the use of machines to replace manual inspection has become increasingly widespread. Vision camera-based inspection mainly relies on passive vision cameras to acquire images of the area in front. These cameras can capture color information of objects, offering fast acquisition speeds and abundant data. However, they have several fatal flaws: firstly, the data acquired by the camera is greatly affected by lighting conditions, such as requiring additional lighting at night and reduced visibility in rainy or foggy weather; secondly, the image data does not contain spatial information, making it difficult to determine the specific spatial relationship between personnel and hazardous areas. LiDAR, on the other hand, is less affected by lighting conditions, contains depth information, and has advantages such as wide detection range and high distance resolution. It is widely used in environmental inspection systems for autonomous driving, ports, and railways. Considering the relatively enclosed environment of coal storage areas, poor nighttime lighting conditions, and the presence of large amounts of dust, vision-based inspection methods are prone to false positives and false negatives. Therefore, this invention chooses a LiDAR-based inspection method.

[0041] Existing lidar-based intrusion detection mechanisms have inherent limitations: they only trigger alarms when an intrusion actually occurs, resulting in significant delays in security response and insufficient time for staff to react. Therefore, predicting potential intrusion threats is more important than developing a real-time alarm system.

[0042] This invention adopts the approach of "offline training and online detection," such as... Figure 1 As shown, a deep learning-based 3D point cloud intrusion detection method is proposed. Before online detection, the EFT-RCNN human object detection network needs to be trained offline on the dataset to implement the human object detection step.

[0043] Next, the first stage of online detection is carried out: extracting point cloud features and inputting the point cloud data into the trained personnel target detection network EFT-RCNN to automatically identify personnel targets; at the same time, the polygonal base of the dangerous area is determined according to the fixed coordinates of the coal yard environment, and the polygonal prism of the coal pile loading area is obtained by combining the height constraint.

[0044] The second stage of intrusion detection, based on online measurement, involves using orthogonal projection to map target personnel and hazardous areas from the same coordinate system onto a two-dimensional plane. This simplifies the intrusion problem into a point-polygon positional relationship analysis within the two-dimensional plane. The ray casting method is then used to determine whether an intrusion point belongs to the L3 alarm layer, while the minimum distance from an intrusion point to the polygon is calculated to determine whether it belongs to the L1 prediction layer or the L2 early warning layer.

[0045] This method takes into account the characteristics of the coal yard environment and makes full use of the spatial relationship between personnel and hazardous areas, which helps to meet the requirements of safety supervision in hazardous areas.

[0046] Specifically, this invention discloses a deep learning-based 3D point cloud intrusion detection method, comprising the following steps: acquiring a point cloud of the scene to be tested; performing target recognition on the point cloud based on a personnel target detection network to obtain personnel target detection information; wherein, the personnel target detection network is constructed based on the Voxel R-CNN framework, and the feature set of the voxel grid is generated according to the position information of the points in the point cloud contained in the voxel grid and the point cloud features; determining whether an intrusion event has occurred based on the detection information and the position information of the dangerous area in the scene to be tested.

[0047] Traditional Voxel R-CNN first uses MeanVoxel Feature Encoding (MeanVFE) to calculate the intensity and average value of the 3D coordinates of the input point cloud as voxel features, completing the voxelization operation. Then, a 3D backbone network is constructed using sparse convolutional blocks to extract voxel features. Next, Height Compression is used to directly compress the 3D voxel features into 2D features at the height, completing the conversion from sparse tensors to a bird's-eye view (BEV). Then, the 2D backbone network is used to extract BEV image features and generate Region of Interest (RoI) proposals. Finally, Voxel RoIPooling is used to fine-tune the RoI proposals using the voxel features extracted by the 3D backbone network, optimizing the feature input detection head and obtaining the detection results.

[0048] like Figure 2As shown, this invention is based on Voxel R-CNN and proposes a novel personnel target detection network, EFT-RCNN, for high-risk working environments in coal pile loading areas: (1) An EnhancedVFE module is proposed in the VFE layer (voxel feature encoding), which optimizes the VFE voxel partitioning, aggregates local and global features, and improves the feature expression quality by utilizing feature information at different levels. (2) The 3D backbone network structure is reconstructed, and Focal Conv blocks are introduced to dynamically weight each voxel feature, improving the foreground and background distinction and providing more effective information for personnel target detection. (3) In the MAP to BEV part, Transformation-Equivariant BEV Pooling (TeBEV Pooling) is used to replace high compression. The TeBEV Pooling method can suppress complex background noise and retain more comprehensive geometric information.

[0049] The VFE layer is a feature encoding layer that performs the computation from point cloud features to voxel features. The baseline network uses the MeanVFE method, which directly divides the data into voxels using cubes of the same size and generates voxel features by calculating the mean of all points in each voxel. If the number of points in a single voxel is small, the mean calculation is still effective, but the features may not be stable enough.

[0050] Therefore, as Figure 3 As shown, this invention proposes a voxel feature encoding module, EnhancedVFE, which compensates for the shortcomings of MeanVFE in feature utilization by calculating the offset and concatenating it with other features. At the same time, it realizes the aggregation of local and global information and uses feature information at different levels to improve the quality of feature representation.

[0051] Enhanced VFE can be divided into two steps:

[0052] (1) Voxelization: For the input point cloud, the point cloud data The i-th point in point cloud data P is represented as p. i =(x i ,y i ,z i feat i ), where N represents the number of points in the point cloud data, x i y i and z i These represent the position information of the i-th point along the x-axis, y-axis, and z-axis in a three-dimensional coordinate system (in this invention, the three-dimensional coordinate system refers to the northeast-northeast coordinate system or the lidar coordinate system), feat iLet R represent the point cloud features of the i-th point. Divide the point cloud into cubes of the same size, assuming the given point cloud range is R = (min... x ,min y ,min z ,max x ,max y ,max z ), min x ,min y ,min z The max represents the coordinates of the minimum point (i.e., the starting point) of the point cloud. x ,max y ,max z The coordinates of the maximum point (i.e., the end point) representing the point cloud extent, and the depth, height, and width of each voxel are (v... D ,v H ,v W If the total number of voxels generated at each coordinate in the 3D voxelization of the entire data is:

[0053]

[0054] The set of points in a single voxel lattice can be represented as:

[0055]

[0056] In the formula, t represents the number of points in a single voxel, and V represents the set of points in a single voxel. This represents a four-dimensional vector space.

[0057] First, a mask is generated based on the number of points in a voxel. This mask sets the voxel grid features that do not contain points to 0. Then, the point (v) corresponding to the average position of all points within the voxel grid is calculated using the scatter_mean method. x ,v y ,v z The voxel center is used as the centroid of the voxel lattice. Finally, the offset f between each point in the voxel lattice and the voxel center is calculated. center And the offset f from physical fitness cluster and these features (i.e. f) center and f cluster ) and original point features (i.e., feature i Cascading is performed to enhance feature representation capabilities:

[0058] f t =concat(feat) i ,f center ,f cluster (3)f tThe set of features representing a voxel lattice.

[0059] In other words, the feature set of the voxel grid consists of the enhanced features of the points within it. The enhanced feature generation method is as follows: calculate the average position of all points in the voxel grid, and take the point corresponding to the average position as the centroid of the voxel grid; calculate the centroid offset between the point and the corresponding centroid; calculate the center offset between the point and the corresponding center; where the center is the center point of the voxel grid; concatenate the point cloud features, center offset, and centroid offset of the point to obtain the enhanced features.

[0060] (2) Feature aggregation, the specific method is as follows:

[0061] The feature set is passed through three PFN layers to obtain the voxel features of the voxel lattice; in the first and second PFN layers, the enhanced features are concatenated with the pooled enhanced features.

[0062] More specifically, the feature set is processed through three PFN layers for feature extraction and aggregation. For each PFN layer, the features undergo linear transformation, normalization, activation, and scatter_max (scatter point maximum / minimum) pooling operations. Notably, in the first and second PFN layers, the original voxel features are concatenated with the pooled voxel features. This not only preserves the local information of the voxel features but also integrates global information, thereby more effectively extracting and representing point cloud features and providing higher-quality feature input for subsequent 3D perception tasks.

[0063] 3D backbone networks are used to further extract and fuse features in a voxelized 3D space to learn higher-level semantic information and spatial features. Three-dimensional sparse convolution is essential for building a 3D backbone network. There are two commonly used convolution methods in 3D sparse convolution: (1) Regular sparse convolution, which calculates the convolution result of all voxels around the input point (within the convolution kernel). Its limitation is that it increases computation and obscures valuable foreground points. (2) Submanifold sparse convolution, which only calculates the convolution result at the sparse point cloud input position, resulting in less computation. Its limitation is that there are no intersecting input points, and the connectivity representation between point clouds is weak.

[0064] Focal sparse convolution extracts denser features from the foreground region while preserving the original features of the background region. Therefore, focal sparse convolution provides more effective features in 3D object detection, which is beneficial for distinguishing foreground and background regions. Thus, this invention uses Focal Conv blocks to reconstruct the 3D backbone network of Voxel R-CNN.

[0065] like Figure 4 As shown, the 3D backbone network based on focal convolution (consisting of multiple focal sparse convolutional layers) consists of an input convolutional layer, four convolutional blocks, and an output convolutional layer. The first three convolutional blocks have Focal Conv blocks integrated at their end.

[0066] As a specific implementation, the voxel features (M, 192) are received as input, where M represents the number of voxel lattices. First, the 192-dimensional features are compressed into 16-dimensional features using sub-convolutional blocks. Then, multi-scale features are extracted through multiple sparse convolutional blocks and sub-convolutional blocks, and downsampling operations of 1×, 2×, 4×, and 8× are performed to increase the number of feature channels. Finally, a multi-scale sparse feature map and a 128-channel feature map are output.

[0067] It is worth noting that this invention integrates a Focal Conv block at the end of the first three convolutional blocks, such as... Figure 4 As shown in (c), it maps the features along the path to 27 channels using a unique SubMConv3d algorithm to predict the importance of each voxel. FocalLoss then weights the features along the path based on the predicted importance, enhancing the representation of important features and suppressing interference from unimportant features. This allows the network to effectively distinguish and extract useful foreground information in complex background environments. Specifically, the main path performs standard sub-convolution operations, while the side branches predict weights p through independent convolutions. * FocalLoss combines both methods to focus on difficult-to-classify samples, improving feature discrimination. The 3D Focal Backbone network, through the combination of sparse convolution and Focal Conv blocks, effectively improves the feature extraction capability of sparse 3D data while maintaining computational efficiency. The cluttered background of the coal pile loading area affects the extraction of foreground personnel features by the target detection network; this design effectively distinguishes and extracts target personnel.

[0068] Map_to_BEV maps 3D voxel features to a 2D feature map from a bird's-eye view, allowing for subsequent feature extraction and object detection using a mature 2D convolutional neural network. The baseline network compresses voxel features along the height dimension and transforms sparse tensors into dense tensors, ultimately obtaining a 2D feature map. However, Voxel R-CNN loses much crucial information during voxel segmentation and the conversion of 3D voxel features into a bird's-eye view. Especially in the scenario of this invention, the foreground target personnel point cloud information is already scarce; direct voxelization leads to feature loss and reduced detection accuracy.

[0069] TeBEV Pooling aims to align and aggregate multi-channel transform-equivalent meta-features into a compact BEV representation. Its core function is to ensure that the detection results are consistent with the transformation of the input point cloud by explicitly modeling the rotation and reflection transformations, while reducing computational complexity.

[0070] Therefore, this invention incorporates the TeBEV Pooling method into the Voxel R-CNN network to achieve high compression operations, such as... Figure 5 As shown, the TeBEVPool module mainly consists of two steps:

[0071] (1) Feature Alignment: Coordinate alignment is performed on voxel features (such as rotated or reflected features) from different transformation channels. Bilinear interpolation maps BEV features under different transformations to the same coordinate system. Specifically, the voxel feature V is first transformed according to different rotation and reflection angles, formally represented as... As shown in formula (4), T b Let b represent the transformation method, and B represent the number of each transformation method. Then, it is compressed along the height dimension into BEV features. As shown in Equation (5), E represents the highly compressed BEV feature. Since the BEV features are obtained under different transformations, they need to be aligned to the same coordinate system. For example, using... Generate a set of scene-level grid points in the lidar coordinate system The transformation operation converts the grid points to the BEV coordinate system, forming a new set of grid points. As shown in Equation (5). Finally, a series of bilinear interpolation I(·,·) is applied to the BEV map to obtain a set of alignment features. As shown in formula (7).

[0072]

[0073]

[0074] (2) Feature aggregation: To improve efficiency, max-pooling is applied to the 2N aligned feature maps to extract the most significant features from the aligned features, generating a lightweight BEV feature map A. * .

[0075] During the implementation of the detection network, intruders may enter the danger zone from any direction. Traditional methods are prone to missing detections due to changes in direction, while TeBEV Pooling, through multi-channel transform feature alignment and aggregation, retains more comprehensive geometric information and can significantly improve the accuracy of personnel detection. At the same time, TeBEV Pooling effectively highlights key personnel features and suppresses complex background noise by aggregating multiple transform features through max pooling.

[0076] In summary, this invention proposes an EFT-RCNN human target detection network based on Voxel R-CNN, specifically for high-risk working environments in coal pile loading areas.

[0077] On the other hand, the delineation of dangerous areas is a prerequisite for intrusion detection. This invention proposes a parametric clipping box method for dangerous area delineation, which is simple and effective. Its core idea is that dangerous areas are constructed jointly using base plane (XOY) polygons and height constraints.

[0078] In other words, determining whether an intrusion event has occurred based on detection information and the location information of dangerous areas in the scene under test includes: calculating the coordinates of the center point of the detection information; determining the closed polygon based on the location information of the dangerous area; and determining whether an intrusion has occurred based on the coordinates of the center point and the closed polygon.

[0079] First, a lidar coordinate system is established with the lidar installation location as the origin, the lidar's orientation towards the corridor bridge as the x-axis, and the left side as the y-axis. Based on on-site survey data, the vertex coordinates of the base polygon in the XOY plane are determined, accurately mapping the planar projection boundary of the actual hazardous area. Second, the height parameters of the coal yard roof are superimposed to form vertical constraints, creating a configurable 3D clipping box (e.g., formed by splicing). This area division method adapts to the horizontal characteristics of the coal yard ground, avoids redundant area extraction algorithms, and supports dynamic adjustment of the base vertex coordinates, allowing for changes in the work area. This method effectively improves the flexibility of intrusion warning model establishment. Based on these methods, 3D information of the hazardous area can be obtained.

[0080] In actual coal storage sites, the ground can be approximated as an ideal horizontal plane, an assumption that provides theoretical feasibility for dimensionality reduction processing of three-dimensional spatial relationships. Based on this, this invention employs orthogonal projection to map target personnel and hazardous areas in the same coordinate system to a two-dimensional plane, such as... Figure 6 As shown.

[0081] Let P be the coordinates of the center point of the 3DBBox (3D detection box) that detects the target person. 3D =(x p ,y p ,z p The 3DBox has dimensions of S. 3D= (w, l, h), the set of base vertices of the 3D polygon of the danger zone is {V j =(x j ,y j ,z j )|j=1,2,3,4}, where j represents the index of the vertex. Since the ground is horizontal, a projection operator is established. We can obtain:

[0082] Π(P 3D )=(x p ,y p ),Π(V j )=(x j ,y j (8)

[0083] The coordinates of the target person point after projection are P 2D =(x p ,y p The danger zone forms a polygon R. This mapping process preserves the topological relationships in three-dimensional space while simplifying the intrusion problem into a point-polygon positional relationship analysis in a two-dimensional plane, significantly reducing the algorithm complexity.

[0084] Based on this, the present invention proposes a personnel intrusion early warning model, which constructs a hierarchical early warning strategy of L1, L2 and L3 through a hierarchical judgment method. First, the ray method is used to judge the internal and external points; second, the minimum distance from the external point to the rectangle R (taking the rectangle as an example) is calculated.

[0085] (1) Determine the positional relationship between the point and the polygon.

[0086] For any closed polygon in a plane, the polygon divides the plane into two regions: inside and outside. For any given straight line in the plane, when it intersects the boundary of the polygon, there are only two possibilities: either it enters the polygon or it moves away from the polygon. Therefore, this invention uses the ray method to determine whether a person target exists in a danger zone (angle method, bisection method, Monte Carlo method, etc. can also be used). The equation determines whether an intrusion has occurred within the polygonal region:

[0087] χ=sum[q(P 2D ,n)%2] (9)

[0088] In the formula, χ represents the number of intersections between the center point coordinates and the closed polygon. When χ is odd, it indicates that an intrusion has occurred; when χ is even, it indicates that no intrusion has occurred. q represents the distance from point P. 2D The number of actual intersections between n rays emitted in any direction and the sides of a rectangle, where %2 represents the remainder after dividing by 2, and sum represents the summation.

[0089] When the number of intersections of the rays is odd, the target personnel are within the danger zone; otherwise, they are outside the danger zone. In a special case, if the point of intersection falls on the boundary of the polygon, it is still classified as L3, and an intrusion occurs.

[0090] (2) Calculation of the shortest external distance.

[0091] When χ is even, calculate the shortest distance between the center point coordinates and the closed polygon, and determine the intrusion level based on the shortest distance and the distance threshold.

[0092] Specifically, when the target personnel are outside the danger zone, the problem is transformed into: finding the shortest distance d from the point to each side of the polygon, which is equivalent to calculating the length of the line segment equation V(t). Finally, by comparing the lengths of the line segment equations, the external point P can be obtained. 2D The shortest distance d to rectangle R min V(t) is represented as:

[0093]

[0094] In the formula: V(t) is the distance line segment from the external point to each side of the polygon; Let A be a vector representing any side of the rectangle, and let A = (x1, y1) and B = (x2, y2); t is the projection parameter, which can be obtained using the dot product formula, and it has three cases representing three different values ​​of V(t), such as... Figure 7 As shown, point C = (x c ,y c ) is P 2D arrive The projection point: when t < 0, C is at On the extended line, the shortest distance is line segment P. 2D A; When 0 ≤ t ≤ 1, C is in Above, at this point, the shortest distance is taken as line segment P. 2D C; when t>1, C is in On the extended line, the shortest distance is P. 2D B. Therefore, d is:

[0095]

[0096] Using the above method, point P can be determined. 2D The shortest distance d to polygon R min That is, the shortest distance from the target personnel to the danger zone.

[0097] In one embodiment, a hierarchical intrusion warning model is established by utilizing the distance relationship between the two: d min >2m is used as the prediction layer (L1); 2m≥d min A depth of ≥0.5m is designated as the early warning layer (L2); 0.5m > d minThe determination that personnel are in a dangerous area serves as the alarm layer (L3).

[0098] Therefore, this invention utilizes the spatial location information of three-dimensional point clouds to construct a dangerous area classification and discrimination model with prediction, early warning, and alarm layers based on the distance relationship between dangerous areas and target pedestrians. This allows for more accurate and longer-term prediction of personnel intrusion threats, thereby preventing accidents from occurring.

[0099] This invention addresses the problem of hazard intrusion detection in coal bunker and coal pile working areas. Taking into account the actual conditions of hazardous areas in coal bunker work sites, the high risk of these areas, and the irregular placement of coal piles, it provides an effective solution for intrusion in coal pile loading areas: "offline training, online detection," thereby improving the safety of mechanical equipment operations in coal pile loading areas.

[0100] To address personnel safety issues in high-risk coal loading areas, this paper proposes using lidar to detect personnel intrusions into these areas. Based on this, a 3D point cloud target detection network and intrusion judgment method are developed. To overcome interference from coal dust and complex backgrounds in coal yards, a two-stage EFT-RCNN detection network is proposed to achieve robust identification of coal yard workers. Based on personnel detection results and a defined danger zone, a three-level intrusion judgment model is constructed. This model progressively analyzes the spatial relationship between personnel and danger zones through a prediction layer, an early warning layer, and an alarm layer, avoiding the limitations of traditional single-step intrusion judgment and providing a systematic approach from target perception to safety decision-making for high-risk work environments.

[0101] In summary, in intrusion detection scenarios in hazardous areas, missed detections and false detections can lead to serious security incidents. EFT-RCNN achieves a good balance between detection accuracy and speed. Its high accuracy under difficult sample conditions can significantly reduce the risk of missed detections and false detections in complex environments; at the same time, its balanced real-time performance can meet the response speed requirements of practical detection systems, making it highly promising for engineering applications.

[0102] The present invention also discloses a deep learning-based 3D point cloud intrusion detection device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement any of the above methods.

[0103] The present invention also discloses an embodiment that provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0104] The present invention also provides a computer program product that, when run on a data storage device, enables the data storage device to implement the steps in the above-described method embodiments.

[0105] If the integrated unit module is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a storage device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0106] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

Claims

1. A deep learning based three-dimensional point cloud intrusion detection method, characterized in that, Includes the following steps: Obtain the point cloud of the scene to be tested; The point cloud is used to identify targets based on a personnel target detection network to obtain personnel target detection information; wherein, the personnel target detection network is constructed based on the Voxel R-CNN framework, and the feature set of the voxel grid is generated according to the position information of the points in the point cloud contained in the voxel grid and the point cloud features; Based on the detection information and the location information of the dangerous area in the scene to be tested, it is determined whether an intrusion event has occurred; The feature set of the voxel lattice consists of enhanced features of its points, and the enhanced feature generation method is as follows: Calculate the average position of all points in the voxel lattice, and take the point corresponding to the average position as the centroid of the voxel lattice; Calculate the centroid offset between the point and its corresponding centroid; Calculate the center offset of the point from the corresponding center; where the center is the center point of the voxel lattice; The enhanced feature is obtained by concatenating the point cloud features, center offset, and centroid offset of the point.

2. The method of claim 1, wherein the method is based on deep learning of three-dimensional point cloud intrusion detection. It also includes feature aggregation of the feature set of the voxel lattice, specifically as follows: The feature set is passed through three PFN layers to obtain the voxel features of the voxel lattice; wherein, in the first PFN layer and the second PFN layer, the enhanced features are concatenated with the pooled enhanced features.

3. The method of claim 1, wherein the method is based on deep learning three-dimensional point cloud intrusion detection. The 3D backbone network of the personnel target detection network includes an input convolutional layer, four convolutional blocks, and an output convolutional layer. The first three convolutional blocks are followed by Focal Conv blocks. 4.The method of claim 1 or 3, wherein, The TeBEVPool module is used to achieve high compression in the personnel target detection network.

5. The method of claim 4, wherein the method is based on deep learning three-dimensional point cloud intrusion detection. Determining whether an intrusion event has occurred based on the detection information and the location information of the dangerous area in the scene to be tested includes: Calculate the coordinates of the center point of the detection information; Determine the closed polygon based on the location information of the danger zone; Whether an intrusion has occurred is determined based on the coordinates of the center point and the closed polygon.

6. The method of claim 5, wherein the method is based on deep learning three-dimensional point cloud intrusion detection. Determining whether an intrusion has occurred based on the coordinates of the center point and the closed polygon includes: ， wherein, represents the number of intersections of the center point coordinate with the closed polygon, when represents an intrusion when odd, and represents no intrusion when even, q represents the center point coordinate is emitted in any direction rays with the actual number of intersections of the edges of the rectangle, 2 represents the remainder of 2, and sum represents the sum.

7. The method of claim 6, wherein the method is based on deep learning of three-dimensional point cloud intrusion detection. When When even, calculate the shortest distance between the center point coordinate and the closed polygon, and determine the intrusion level according to the shortest distance and a distance threshold.

8. A three-dimensional point cloud intrusion detection device based on deep learning, comprising a memory, a processor and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-7.

Images