A Deep Learning-Based 3D Security Sensor Early Warning and Protection Method

Abstract

This invention discloses a 3D security sensor early warning and protection method based on deep learning, comprising the following steps: S1, acquiring a 3D sensor point cloud frame sequence; S2, constructing a spatial reachability function based on security protection rules, and labeling the reachable point set frame by frame in the point cloud; S3, performing KPConv kernel convolution under reachability constraints to extract 3D features restricted by security boundaries; S4, aggregating the 3D features by spatial units to generate a frame-level structure embedding vector sequence; S5, inputting the structure embedding vector into a DAGMM network, calculating the anomaly energy value and filtering anomaly candidates; S6, performing anomaly reversibility testing on the anomaly candidates, and obtaining the minimum perturbation cost through constrained reverse optimization; S7, based on the joint determination of the anomaly energy value and the minimum perturbation cost, outputting the early warning level and the set of early warning events. This invention achieves accurate identification and graded early warning of 3D security anomalies, reduces false alarms, and improves early warning stability.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional sensor data processing technology, and in particular to a 3D security protection sensor early warning and protection method based on deep learning. Background Technology

[0002] With the widespread application of 3D sensors such as 3D LiDAR and depth cameras, spatial environment perception and security protection technologies based on point cloud data have been gradually applied to scenarios such as industrial production, safety supervision, and intelligent site management. Existing technologies typically monitor the status of personnel, equipment, or obstacles by performing spatial modeling or target detection on 3D point cloud data, and then use preset rules or thresholds to determine anomalies and issue alarms. Among these methods, one type focuses on spatial determination based on geometric features or rule constraints, while another type introduces deep learning models to model and analyze point clouds or their derived features to improve perception capabilities in complex scenes.

[0003] Existing technologies still have significant shortcomings. Traditional 3D security protection methods based on rules or geometric constraints typically only filter or partition point clouds at the coordinate level, making it difficult to incorporate security semantic information such as spatial accessibility and regional constraints during the feature extraction stage. This results in insufficient perception of security boundaries and propagation limitations by deep features. Existing deep learning-based anomaly detection methods mostly rely on a single anomaly scoring index for judgment, lacking further analysis of the "recoverability" or "reversibility" of anomaly states. This easily leads to the mixing of short-term disturbances and recoverable behaviors with truly high-risk anomalies, reducing the accuracy and reliability of early warning results.

[0004] Therefore, how to provide a 3D security protection sensor early warning and protection method based on deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a 3D security protection sensor early warning and protection method based on deep learning. This invention utilizes 3D point cloud processing technology and deep learning anomaly detection method. By introducing spatial reachability constraints, kernel point convolution feature modeling, and anomaly reversibility testing, it achieves accurate identification and hierarchical early warning of abnormal states in complex spatial environments. It has the advantages of high accuracy in early warning judgment, strong risk differentiation ability, and wide applicability.

[0006] A 3D security sensor early warning and protection method based on deep learning according to an embodiment of the present invention includes the following steps:

[0007] S1. Obtain the point cloud frame sequence and timestamp sequence collected by the 3D sensor, perform preprocessing, and obtain a point cloud frame sequence in a unified coordinate system.

[0008] S2. Obtain the set of security protection rules, construct the spatial reachability function, and label the point cloud frame sequence in the unified coordinate system frame by frame according to the spatial reachability function to obtain the reachable point set sequence;

[0009] S3. Construct a neighborhood point set within the reachable point set sequence, prohibiting the neighborhood point set from crossing the non-propagable boundary defined by the spatial reachability function, and perform KPConv kernel point convolution to obtain a three-dimensional feature sequence;

[0010] S4. Perform aggregation processing on the three-dimensional feature sequence to obtain the frame-level structure embedding vector sequence;

[0011] S5. Construct a DAGMM network, input the frame-level structure embedding vector sequence, perform encoding, decoding and reconstruction error calculation, obtain the anomaly description vector sequence and calculate the anomaly energy value sequence, and filter the anomaly candidate sequence according to the anomaly energy value threshold.

[0012] S6. Perform anomaly reversibility test on the anomalous candidate sequence, fix the DAGMM network parameters, perform restricted inverse optimization on the latent representation corresponding to the anomalous candidate sequence, and obtain the minimum perturbation cost sequence that falls back to the normal Gaussian component.

[0013] S7. Output the early warning level sequence and early warning event set based on the abnormal energy value sequence and the minimum disturbance cost sequence.

[0014] Optionally, S2 specifically includes:

[0015] S21. Obtain the set of security protection rules and convert the set of security protection rules into spatial constraint data under the unified coordinate system of the protection area. The spatial constraint data limits the allowed and prohibited spatial ranges within the protection area.

[0016] S22. Based on the spatial constraint data, construct a spatial reachability function in the unified coordinate system of the protected area. The spatial reachability function takes spatial coordinates as input and determines whether the spatial coordinates are within the allowable reachable space based on the spatial constraint data.

[0017] S23. Divide the unified coordinate system of the protected area into spatial units according to the preset spatial resolution, generate a set of discrete spatial units, and call the spatial reachability function for each spatial unit to calculate the corresponding reachability identifier value.

[0018] S24. Using the unified coordinate system point cloud frame sequence as input, call the spatial reachability mapping relationship for each point in each frame of the point cloud to obtain the corresponding reachability identifier value;

[0019] S25. Based on the reachability identifier value, filter the points in each frame of the point cloud, and form a reachable point set for the points whose reachability identifier value meets the reachability condition, generating a sequence of reachable point sets arranged in chronological order.

[0020] Optionally, S3 specifically includes:

[0021] S31. Using the unified coordinate system point cloud frame sequence and the corresponding reachability identifier value as input, determine the spatial coordinates and reachability identifier value for each point in each frame of the point cloud.

[0022] S32. Using each point as the center point of the convolution, construct a set of neighborhood points in the same frame of the point cloud according to the preset spatial neighborhood search rules. The set of neighborhood points is determined only based on spatial distance relationships.

[0023] S33. Generate the corresponding convolution activation control value based on the reachability flag value of the convolution center point;

[0024] S34. When performing KPConv kernel convolution operation, the convolution activation control value is introduced into the kernel convolution calculation process, and the kernel convolution output at the center point of the convolution is gated and modulated.

[0025] S35. Based on the results of the kernel convolution operation after gated modulation, generate the three-dimensional feature vector corresponding to the convolution center point;

[0026] S36. Repeat S32 to S35 for all points in the same frame point cloud to form a three-dimensional feature sequence that corresponds one-to-one with the point cloud frame sequence, and retain the reachability identifier value corresponding to the point in the three-dimensional feature sequence.

[0027] Optionally, S4 specifically includes:

[0028] S41. Obtain the three-dimensional feature sequence and read the three-dimensional feature set corresponding to each frame in chronological order;

[0029] S42. Based on the preset spatial resolution, the three-dimensional feature vectors in the same frame are mapped to the corresponding spatial unit index numbers according to the spatial coordinates, generating a feature subset corresponding to several spatial units.

[0030] S43. Perform feature accumulation and normalization operations on the feature subsets corresponding to the index number of each spatial unit in the same frame to generate the sub-region embedding vector set of the corresponding spatial unit.

[0031] S44. Arrange the set of sub-region embedding vectors generated within the same frame according to the spatial unit index number and time order to form a frame-level structure embedding vector sequence.

[0032] Optionally, S5 specifically includes:

[0033] S51. Obtain the frame-level structure embedding vector sequence and feed the frame-level structure embedding vector sequence as input into the DAGMM network.

[0034] S52. The input vector sequence is encoded and decoded using a DAGMM network to generate the corresponding latent representation vector and reconstruction vector, and the reconstruction error is calculated based on the input vector and reconstruction vector.

[0035] S53. Combine the latent representation vector and the reconstruction error to form an anomaly description vector, and calculate the anomaly energy value sequence based on the anomaly description vector;

[0036] S54. Filter abnormal candidate sequences based on abnormal energy value threshold.

[0037] Optionally, S6 specifically includes:

[0038] S61. While keeping the DAGMM network parameters fixed, use the latent representation vector corresponding to each anomalous candidate in the anomalous candidate sequence as the optimization variable to initialize the restricted reverse optimization process.

[0039] S62. Using the potential representation vector falling back to the normal Gaussian mixture component determination region as the optimization constraint, perform iterative update operation on the potential representation vector and apply perturbation adjustment to the potential representation vector;

[0040] S63. After each iteration update, perform a normality determination on the updated latent representation vector. The iteration terminates when the updated latent representation vector satisfies the normal Gaussian mixture component determination condition.

[0041] S64. Record the perturbation vector that makes the potential representation vector first satisfy the normal Gaussian mixture component determination condition, and calculate the norm of the perturbation vector as the minimum perturbation cost of the corresponding anomalous candidate.

[0042] S65. Repeat S61 to S64 for all abnormal candidates in the abnormal candidate sequence to form a minimum perturbation cost sequence that corresponds one-to-one with the abnormal candidate sequence.

[0043] Optionally, the restricted inverse optimization in S6 specifically includes:

[0044] The potential representation vector corresponding to the abnormal candidate is used as the initial state vector. The current state vector is set to be equal to the initial state vector, the iteration count is set to zero, and the perturbation vector is initialized to a zero vector with the same dimension as the potential representation vector.

[0045] While keeping the DAGMM network parameters fixed, the abnormal energy value corresponding to the current state vector is calculated based on the Gaussian mixture model in the DAGMM network, and the difference between the abnormal energy value and the preset abnormal energy value threshold is calculated.

[0046] Calculate the gradient vector of the abnormal energy value with respect to the current state vector based on the current state vector, and take the opposite direction of the gradient vector as the direction of the perturbation increment vector;

[0047] The amplitude of the disturbance increment vector is determined based on the difference between the abnormal energy value and the preset abnormal energy value threshold. The amplitude is obtained by multiplying the difference by the preset scaling factor, and amplitude normalization is performed on the disturbance increment vector.

[0048] The perturbation increment vector, after amplitude determination and normalization, is superimposed on the current state vector to obtain the updated potential representation vector, and the perturbation vector is updated to the vector sum of the historical perturbation vector and the perturbation increment vector.

[0049] The corresponding abnormal energy value is calculated based on the updated latent representation vector. When the abnormal energy value is less than the preset abnormal energy value threshold, it is determined that the updated latent representation vector satisfies the normal Gaussian mixture component determination condition and the iteration is terminated.

[0050] When the updated latent representation vector does not meet the normal Gaussian mixture component determination condition, the updated latent representation vector is set as the new current state vector, the iteration count is incremented by one, and the iteration is terminated when the iteration count reaches the preset iteration termination condition.

[0051] Optionally, S7 specifically includes:

[0052] S71. Obtain the abnormal energy value sequence and the minimum disturbance cost sequence, and match the abnormal energy value and the minimum disturbance cost one by one according to the index relationship of the abnormal candidates in the time series to form a joint judgment data sequence of abnormal candidates.

[0053] S72. Based on a pre-set set of early warning thresholds, perform interval judgment processing on the abnormal energy value and minimum disturbance cost in the abnormal candidate joint judgment data sequence, map the abnormal energy value to an energy level identifier, and map the minimum disturbance cost to a reversibility level identifier.

[0054] S73. Based on the energy level identifier and the reversibility level identifier, execute the joint mapping rule to determine the corresponding early warning level for each anomaly candidate, and associate the early warning level with the time index and spatial unit index number corresponding to the anomaly candidate to generate an early warning event record.

[0055] S74. Summarize and output the generated early warning event records to form a set of early warning events containing early warning level, time information and spatial location information, which serves as the early warning output result of the 3D safety protection sensor.

[0056] The beneficial effects of this invention are:

[0057] (1) In the three-dimensional point cloud feature modeling stage, the present invention introduces a spatial reachability function to annotate the point cloud data point by point and constrain the feature propagation during the kernel convolution process, so that the deep feature extraction process can explicitly perceive the reachable boundary and non-propagable area within the protection area, avoid security semantic aliasing caused by unconstrained convolution, and improve the consistency and stability of the expression of three-dimensional features to spatial security rules.

[0058] (2) In the anomaly detection stage, the present invention introduces an anomaly reversibility test mechanism. Under the premise of keeping the anomaly detection model parameters fixed, the minimum disturbance cost required for the anomaly state to fall back to the normal distribution is calculated by restricted reverse optimization. The anomaly intensity and anomaly recoverability are decoupled and characterized, and the short-term disturbance anomaly and high-risk irreversible anomaly are distinguished, thereby improving the precision of anomaly judgment.

[0059] (3) The present invention outputs the warning level and warning event based on the joint judgment method of abnormal energy value and minimum disturbance cost, realizes the hierarchical warning and spatial positioning of abnormal state, reduces the false alarm and missed alarm problem caused by single threshold judgment, and improves the reliability and practical application value of the three-dimensional security protection warning results. Attached Figure Description

[0060] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0061] Figure 1 This is a flowchart of a 3D security protection sensor early warning and protection method based on deep learning proposed in this invention;

[0062] Figure 2 This is a flowchart illustrating the improved KPConv reachability identifier construction process of a deep learning-based 3D security sensor early warning and protection method proposed in this invention.

[0063] Figure 3 This is a constrained inverse optimization structure diagram in the improved DAGMM of the 3D security protection sensor early warning and protection method based on deep learning proposed in this invention. Detailed Implementation

[0064] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0065] refer to Figure 1-3 A 3D security sensor early warning and protection method based on deep learning includes the following steps:

[0066] S1. Obtain the point cloud frame sequence and timestamp sequence collected by the 3D sensor, perform preprocessing, and obtain a point cloud frame sequence in a unified coordinate system.

[0067] S2. Obtain the set of security protection rules, construct the spatial reachability function, and label the point cloud frame sequence in the unified coordinate system frame by frame according to the spatial reachability function to obtain the reachable point set sequence;

[0068] S3. Construct a neighborhood point set within the reachable point set sequence, prohibiting the neighborhood point set from crossing the non-propagable boundary defined by the spatial reachability function, and perform KPConv kernel point convolution to obtain a three-dimensional feature sequence;

[0069] S4. Perform aggregation processing on the three-dimensional feature sequence to obtain the frame-level structure embedding vector sequence;

[0070] S5. Construct a DAGMM network, input the frame-level structure embedding vector sequence, perform encoding, decoding and reconstruction error calculation, obtain the anomaly description vector sequence and calculate the anomaly energy value sequence, and filter the anomaly candidate sequence according to the anomaly energy value threshold.

[0071] S6. Perform anomaly reversibility test on the anomalous candidate sequence, fix the DAGMM network parameters, perform restricted inverse optimization on the latent representation corresponding to the anomalous candidate sequence, and obtain the minimum perturbation cost sequence that falls back to the normal Gaussian component.

[0072] S7. Output the early warning level sequence and early warning event set based on the abnormal energy value sequence and the minimum disturbance cost sequence.

[0073] In this embodiment, step S1 specifically includes the following steps:

[0074] S11. Acquire point cloud frame data and corresponding timestamp information continuously collected by the three-dimensional sensor to form the original point cloud frame sequence and timestamp sequence.

[0075] S12. Based on the timestamp sequence, sort and time-align the original point cloud frame sequence to obtain a point cloud frame sequence with consistent time.

[0076] S13. Transform the coordinates of each point in the time-consistent point cloud frame sequence from the sensor coordinate system to the preset unified coordinate system of the protected area to generate a unified coordinate system point cloud frame sequence.

[0077] S14. Perform noise point removal processing on the point cloud frame sequence of the unified coordinate system frame by frame to obtain the denoised point cloud frame sequence.

[0078] S15. Perform point cloud density normalization processing on the denoised point cloud frame sequence to generate a point cloud frame sequence with a unified coordinate system.

[0079] In this embodiment, S2 specifically includes:

[0080] S21. Obtain the set of security protection rules and convert the set of security protection rules into spatial constraint data under the unified coordinate system of the protection area. The spatial constraint data limits the allowed and prohibited spatial ranges within the protection area.

[0081] S22. Based on spatial constraint data, construct a spatial accessibility function in a unified coordinate system of the protected area. The spatial accessibility function takes spatial coordinates as input and determines whether the spatial coordinates are within the allowable reachable space based on the spatial constraint data; and performs coordinate inclusion judgment through spatial constraint data.

[0082] S23. Divide the unified coordinate system of the protected area into spatial units according to the preset spatial resolution, generate a set of discrete spatial units, and call the spatial reachability function for each spatial unit to calculate the corresponding reachability identifier value; use the center coordinates as the position coordinates for each spatial unit.

[0083] S24. Using the unified coordinate system point cloud frame sequence as input, call the spatial reachability mapping relationship for each point in each frame of the point cloud to obtain the corresponding reachability identifier value;

[0084] S25. Based on the reachability identifier value, filter the points in each frame of the point cloud, and form a reachable point set for the points whose reachability identifier value meets the reachability condition, generating a sequence of reachable point sets arranged in chronological order.

[0085] In this embodiment, S3 specifically includes:

[0086] S31. Using the unified coordinate system point cloud frame sequence and the corresponding reachability identifier value as input, determine the spatial coordinates and reachability identifier value for each point in each frame of the point cloud.

[0087] S32. Using each point as the convolution center point, construct a neighborhood point set in the same frame point cloud according to a preset spatial neighborhood search rule. The neighborhood point set is determined only based on spatial distance relationships. Using the spatial coordinates of the convolution center point as a reference, retrieve points in the same frame point cloud whose spatial distance is not greater than the search radius according to a preset search radius as candidate neighborhood points. When the number of candidate neighborhood points exceeds a preset threshold, select a preset number of points in order of spatial distance from near to far to form a neighborhood point set. In this embodiment, the distance is 0.5 meters.

[0088] S33. Generate corresponding convolution activation control values ​​based on the reachability flag value of the convolution center point; in this embodiment, the convolution activation control value for the reachability flag is set to 1 and the convolution activation control value for the inaccessibility flag is set to 0.3.

[0089] S34. When performing KPConv kernel convolution operation, the convolution activation control value is introduced into the kernel convolution calculation process, and the kernel convolution output at the center point of the convolution is gated and modulated; KPConv calculates the original convolution output feature vector normally; and the original convolution output feature vector is multiplied element-wise with the activation control value.

[0090] S35. Based on the results of the kernel convolution operation after gated modulation, generate the three-dimensional feature vector corresponding to the convolution center point;

[0091] S36. Repeat S32 to S35 for all points in the same frame point cloud to form a three-dimensional feature sequence that corresponds one-to-one with the point cloud frame sequence, and retain the reachability identifier value corresponding to the point in the three-dimensional feature sequence.

[0092] In this embodiment, S4 specifically includes:

[0093] S41. Obtain the three-dimensional feature sequence and read the three-dimensional feature set corresponding to each frame in chronological order;

[0094] S42. Based on a preset spatial resolution, the three-dimensional feature vectors within the same frame are mapped to their corresponding spatial unit index numbers according to their spatial coordinates, generating several feature subsets corresponding to the spatial units; in this embodiment, the resolution is 0.5×0.5×0.5m. 3 ;

[0095] S43. Perform feature accumulation and normalization operations on the feature subsets corresponding to the index number of each spatial unit in the same frame to generate the sub-region embedding vector set of the corresponding spatial unit.

[0096] S44. Arrange the set of sub-region embedding vectors generated within the same frame according to the spatial unit index number and time order to form a frame-level structure embedding vector sequence.

[0097] In this embodiment, S5 specifically includes:

[0098] S51. Obtain the frame-level structure embedding vector sequence and feed the frame-level structure embedding vector sequence as input into the DAGMM network.

[0099] S52. The input vector sequence is encoded and decoded using a DAGMM network to generate the corresponding latent representation vector and reconstruction vector, and the reconstruction error is calculated based on the input vector and reconstruction vector.

[0100] S53. The latent representation vector and the reconstruction error are combined to form an anomaly description vector, and the anomaly energy value sequence is calculated based on the anomaly description vector; in this embodiment, the anomaly description vector is obtained by concatenating the latent representation vector and the reconstruction error.

[0101] S54. Screening abnormal candidate sequences based on an abnormal energy value threshold. In this embodiment, the abnormal energy value threshold is set to 95%; when the abnormal energy value corresponding to the sample to be detected is greater than the abnormal energy value threshold, the sample is marked as an abnormal candidate sample.

[0102] In this embodiment, S6 specifically includes:

[0103] S61. While keeping the DAGMM network parameters fixed, the latent representation vector corresponding to each anomalous candidate in the anomalous candidate sequence is used as the optimization variable to initialize the restricted backward optimization process; the restricted backward optimization uses the original latent representation vector corresponding to the anomalous candidate as the initial optimization state.

[0104] S62. Using the potential representation vector falling back to the normal Gaussian mixture component determination region as the optimization constraint, perform iterative update operation on the potential representation vector and apply perturbation adjustment to the potential representation vector;

[0105] S63. After each iteration update, a normality determination is performed on the updated potential representation vector. The iteration is terminated when the updated potential representation vector meets the normal Gaussian mixture component determination condition. In this embodiment, for the potential representation vector obtained in the restricted back optimization process, the corresponding abnormal energy value is calculated based on the Gaussian mixture model in the DAGMM network. When the abnormal energy value is less than the preset abnormal energy value threshold, it is determined that the potential representation vector falls back to the normal Gaussian mixture component determination region.

[0106] S64. Record the perturbation vector that makes the potential representation vector first satisfy the normal Gaussian mixture component determination condition, and calculate the norm of the perturbation vector as the minimum perturbation cost of the corresponding anomalous candidate; the norm of the perturbation vector is calculated using the Euclidean norm.

[0107] S65. Repeat S61 to S64 for all abnormal candidates in the abnormal candidate sequence to form a minimum perturbation cost sequence that corresponds one-to-one with the abnormal candidate sequence.

[0108] In this embodiment, the restricted reverse optimization in S6 specifically includes:

[0109] The potential representation vector corresponding to the abnormal candidate is used as the initial state vector. The current state vector is set to be equal to the initial state vector, the iteration count is set to zero, and the perturbation vector is initialized to a zero vector with the same dimension as the potential representation vector.

[0110] While keeping the DAGMM network parameters fixed, the abnormal energy value corresponding to the current state vector is calculated based on the Gaussian mixture model in the DAGMM network, and the difference between the abnormal energy value and the preset abnormal energy value threshold is calculated.

[0111] Calculate the gradient vector of the abnormal energy value with respect to the current state vector based on the current state vector, and take the opposite direction of the gradient vector as the direction of the perturbation increment vector;

[0112] The amplitude of the disturbance increment vector is determined based on the difference between the abnormal energy value and the preset abnormal energy value threshold. The amplitude is obtained by multiplying the difference by the preset scaling factor 0.1, and amplitude normalization is performed on the disturbance increment vector.

[0113] The perturbation increment vector, after amplitude determination and normalization, is superimposed on the current state vector to obtain the updated potential representation vector, and the perturbation vector is updated to the vector sum of the historical perturbation vector and the perturbation increment vector.

[0114] The corresponding anomalous energy value is calculated based on the updated latent representation vector. When the anomalous energy value is less than a preset anomalous energy value threshold, the updated latent representation vector is determined to meet the normal Gaussian mixture component determination condition, and the iteration is terminated. The condition remains the same as before, at 95%.

[0115] If the updated latent representation vector does not meet the criteria for normal Gaussian mixture components, the updated latent representation vector is set as the new current state vector, the iteration count is incremented, and the iteration terminates when the iteration count reaches the preset termination condition. The maximum number of iterations is set to 50; the convergence threshold is 10. -8 .

[0116] In this embodiment, S7 specifically includes:

[0117] S71. Obtain the abnormal energy value sequence and the minimum disturbance cost sequence, and match the abnormal energy value and the minimum disturbance cost one by one according to the index relationship of the abnormal candidates in the time series to form a joint judgment data sequence of abnormal candidates.

[0118] S72. Based on a pre-set set of early warning thresholds, perform interval judgment processing on the abnormal energy value and minimum disturbance cost in the abnormal candidate joint judgment data sequence, map the abnormal energy value to an energy level identifier, and map the minimum disturbance cost to a reversibility level identifier.

[0119] In this embodiment, the interval discrimination process is as follows: Based on normal state samples, the distribution of abnormal energy values ​​is calculated. A first threshold for abnormal energy values ​​is set as the 95th percentile, and a second threshold is set as the 99th percentile. Interval discrimination processing is performed on the abnormal energy values ​​corresponding to abnormal candidates: when the abnormal energy value is less than the first threshold, the corresponding abnormal candidate is mapped to a low energy level; when the abnormal energy value is greater than or equal to the first threshold and less than the second threshold, the corresponding abnormal candidate is mapped to a medium energy level; when the abnormal energy value is greater than or equal to the second threshold, the corresponding abnormal candidate is mapped to a high energy level. A first threshold for the minimum perturbation cost is set to 0.5, and a second threshold for the minimum perturbation cost is set to 1.5. Interval discrimination processing is performed on the minimum perturbation cost corresponding to the abnormal candidate: when the minimum perturbation cost is less than the first threshold, the corresponding abnormal candidate is mapped to a high reversibility level; when the minimum perturbation cost is greater than or equal to the first threshold and less than the second threshold, the corresponding abnormal candidate is mapped to a medium reversibility level; when the minimum perturbation cost is greater than or equal to the second threshold, the corresponding abnormal candidate is mapped to a low reversibility level.

[0120] S73. Based on the energy level identifier and the reversibility level identifier, execute the joint mapping rule to determine the corresponding early warning level for each anomaly candidate, and associate the early warning level with the time index and spatial unit index number corresponding to the anomaly candidate to generate an early warning event record.

[0121] S74. Summarize and output the generated early warning event records to form a set of early warning events containing early warning level, time information and spatial location information, which serves as the early warning output result of the 3D safety protection sensor.

[0122] Example 1: To verify the feasibility of this invention in practice, it was applied to a three-dimensional security protection scenario with multi-regional spatial constraints, overlapping personnel and equipment activities, and dynamically changing environmental conditions. In this scenario, three-dimensional sensors continuously acquire spatial point cloud data. Permissible activity areas and restricted access areas exist simultaneously within the space, and the local spatial state changes over time. Existing three-dimensional security protection methods often rely on a single anomaly detection index. When short-term occlusion, local density changes, or transient movements occur, recoverable states are easily misjudged as high-risk anomalies, leading to frequent false alarms, significant fluctuations in warning levels, and affecting the long-term stable operation of the security protection system.

[0123] In this embodiment, the system first performs time alignment, coordinate unification, and noise suppression on the continuously acquired point cloud frame sequence to form a stable point cloud sequence with a unified coordinate system. Then, based on preset security protection rules, a spatial reachability function is constructed to label the reachability of each point in the point cloud, ensuring that spatial security constraints are explicitly expressed at the data level. During the feature extraction stage, kernel convolution with reachability gating is introduced to constrain feature propagation within the point cloud neighborhood, preventing unreasonable feature fusion between different security semantic regions and improving the consistency of 3D features in representing spatial structure and security boundaries.

[0124] After completing the 3D feature extraction, a frame-level structure embedding vector sequence is generated through spatial unit aggregation and input into the anomaly detection network for modeling and analysis. In the anomaly detection phase, anomaly energy values ​​are calculated based on the frame-level structure embedding vectors to measure the degree to which the current state deviates from the normal distribution, and anomaly candidate states are selected accordingly. For anomaly candidate states, this invention further introduces an anomaly reversibility testing mechanism. While keeping the anomaly detection model parameters constant, constrained backpropagation is used to calculate the minimum perturbation cost required for the anomaly state to fall back to the normal distribution, thus characterizing the recoverability of the anomaly state.

[0125] In actual operation, the system outputs tiered early warning information based on the joint determination result of abnormal energy value and minimum disturbance cost. For states with high abnormal energy value but low minimum disturbance cost, the system determines it to be a short-term disturbance anomaly and outputs only a low or medium level warning; for states with both high abnormal energy value and high minimum disturbance cost, the system determines it to be a persistent risk state and outputs a high level warning. This joint determination method reduces unnecessary warning triggering and improves the stability and reliability of the overall warning results.

[0126] To verify the practical effectiveness of the method of this invention, a comparative analysis was conducted with a method that does not introduce spatial reachability constraints and anomaly reversibility testing. The continuous operation results show that the present invention demonstrates advantages in anomaly identification accuracy, false alarm control capability, and early warning stability. Especially in situations with frequent short-term disturbances, it can reduce invalid early warnings and frequent jumps in early warning levels.

[0127] Table 1: Comparison of the effectiveness of safety early warning methods

[0128] Comparison Methods Anomaly detection accuracy False alarm rate Warning trigger frequency Number of fluctuations in warning level Early warning stability indicators Traditional anomaly detection methods 0.84 0.19 126 42 0.66 Anomaly detection methods with spatial constraints 0.89 0.14 98 28 0.75 Method of the present invention 0.94 0.07 61 11 0.88

[0129] The comparison results shown in Table 1 reveal differences in anomaly identification capabilities and early warning output behavior among different methods. Traditional anomaly detection methods, relying solely on a single anomaly criterion for early warning decisions, tend to misclassify recoverable states as anomalies when short-term disturbances or local changes occur in the spatial state. This leads to a high false alarm rate, high frequency of early warning triggers, and frequent fluctuations in early warning levels, resulting in unstable early warning outcomes. The comparative methods incorporating spatial constraints improve the ability of feature representations to perceive spatial structure to some extent, increasing anomaly identification accuracy and reducing false alarm rates and early warning trigger frequencies. However, they still lack characterization of anomaly recoverability during the anomaly identification stage, and early warning levels still exhibit fluctuations.

[0130] The method of this invention further improves the accuracy of anomaly identification and reduces the false alarm rate, indicating that it can more effectively distinguish between genuine anomalies and short-term disturbances. The significant reduction in the frequency of warning triggers and the number of fluctuations in warning levels demonstrates that this invention is more restrained and stable in outputting warning results, avoiding repeated warnings and frequent level switching. The improved warning stability index reflects that by jointly introducing spatial reachability constraints and anomaly reversibility testing mechanisms, anomaly judgment is transformed from a single intensity judgment to a multi-dimensional comprehensive analysis, thereby achieving more reliable, continuous, and practically valuable warning output in complex three-dimensional security protection scenarios.

[0131] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A 3D security protection sensor early warning and protection method based on deep learning, characterized in that, Includes the following steps: S1. Obtain the point cloud frame sequence and timestamp sequence collected by the 3D sensor, perform preprocessing, and obtain a point cloud frame sequence in a unified coordinate system. S2. Obtain the set of security protection rules, construct the spatial reachability function, and label the point cloud frame sequence in the unified coordinate system frame by frame according to the spatial reachability function to obtain the reachable point set sequence; S3. Construct a neighborhood point set within the reachable point set sequence, prohibiting the neighborhood point set from crossing the non-propagable boundary defined by the spatial reachability function, and perform KPConv kernel point convolution to obtain a three-dimensional feature sequence; S4. Perform aggregation processing on the three-dimensional feature sequence to obtain the frame-level structure embedding vector sequence; S5. Construct a DAGMM network, input the frame-level structure embedding vector sequence, perform encoding, decoding and reconstruction error calculation, obtain the anomaly description vector sequence and calculate the anomaly energy value sequence, and filter the anomaly candidate sequence according to the anomaly energy value threshold. S6. Perform anomaly reversibility test on the anomalous candidate sequence, fix the DAGMM network parameters, perform restricted inverse optimization on the latent representation corresponding to the anomalous candidate sequence, and obtain the minimum perturbation cost sequence that falls back to the normal Gaussian component. S7. Output the early warning level sequence and early warning event set based on the abnormal energy value sequence and the minimum disturbance cost sequence.

2. The 3D security protection sensor early warning and protection method based on deep learning according to claim 1, characterized in that, S2 specifically includes: S21. Obtain the set of security protection rules and convert the set of security protection rules into spatial constraint data under the unified coordinate system of the protection area. The spatial constraint data limits the allowed and prohibited spatial ranges within the protection area. S22. Based on the spatial constraint data, construct a spatial reachability function in the unified coordinate system of the protected area. The spatial reachability function takes spatial coordinates as input and determines whether the spatial coordinates are within the allowable reachable space based on the spatial constraint data. S23. Divide the unified coordinate system of the protected area into spatial units according to the preset spatial resolution, generate a set of discrete spatial units, and call the spatial reachability function for each spatial unit to calculate the corresponding reachability identifier value. S24. Using the unified coordinate system point cloud frame sequence as input, call the spatial reachability mapping relationship for each point in each frame of the point cloud to obtain the corresponding reachability identifier value; S25. Based on the reachability identifier value, filter the points in each frame of the point cloud, and form a reachable point set for the points whose reachability identifier value meets the reachability condition, generating a sequence of reachable point sets arranged in chronological order.

3. The 3D security protection sensor early warning and protection method based on deep learning according to claim 2, characterized in that, S3 specifically includes: S31. Using the unified coordinate system point cloud frame sequence and the corresponding reachability identifier value as input, determine the spatial coordinates and reachability identifier value for each point in each frame of the point cloud. S32. Using each point as the center point of the convolution, construct a set of neighborhood points in the same frame of the point cloud according to the preset spatial neighborhood search rules. The set of neighborhood points is determined only based on spatial distance relationships. S33. Generate the corresponding convolution activation control value based on the reachability flag value of the convolution center point; S34. When performing KPConv kernel convolution operation, the convolution activation control value is introduced into the kernel convolution calculation process, and the kernel convolution output at the center point of the convolution is gated and modulated. S35. Based on the results of the kernel convolution operation after gated modulation, generate the three-dimensional feature vector corresponding to the convolution center point; S36. Repeat S32 to S35 for all points in the same frame point cloud to form a three-dimensional feature sequence that corresponds one-to-one with the point cloud frame sequence, and retain the reachability identifier value corresponding to the point in the three-dimensional feature sequence.

4. The 3D security protection sensor early warning and protection method based on deep learning according to claim 3, characterized in that, S4 specifically includes: S41. Obtain the three-dimensional feature sequence and read the three-dimensional feature set corresponding to each frame in chronological order; S42. Based on the preset spatial resolution, the three-dimensional feature vectors in the same frame are mapped to the corresponding spatial unit index numbers according to the spatial coordinates, generating a feature subset corresponding to several spatial units. S43. Perform feature accumulation and normalization operations on the feature subsets corresponding to the index number of each spatial unit in the same frame to generate the sub-region embedding vector set of the corresponding spatial unit. S44. Arrange the set of sub-region embedding vectors generated within the same frame according to the spatial unit index number and time order to form a frame-level structure embedding vector sequence.

5. The 3D security protection sensor early warning and protection method based on deep learning according to claim 4, characterized in that, S5 specifically includes: S51. Obtain the frame-level structure embedding vector sequence and feed the frame-level structure embedding vector sequence as input into the DAGMM network. S52. The input vector sequence is encoded and decoded using a DAGMM network to generate the corresponding latent representation vector and reconstruction vector, and the reconstruction error is calculated based on the input vector and reconstruction vector. S53. Combine the latent representation vector and the reconstruction error to form an anomaly description vector, and calculate the anomaly energy value sequence based on the anomaly description vector; S54. Filter abnormal candidate sequences based on abnormal energy value threshold.

6. The 3D security protection sensor early warning and protection method based on deep learning according to claim 5, characterized in that, S6 specifically includes: S61. While keeping the DAGMM network parameters fixed, use the latent representation vector corresponding to each anomalous candidate in the anomalous candidate sequence as the optimization variable to initialize the restricted reverse optimization process. S62. Using the potential representation vector falling back to the normal Gaussian mixture component determination region as the optimization constraint, perform iterative update operation on the potential representation vector and apply perturbation adjustment to the potential representation vector; S63. After each iteration update, perform a normality determination on the updated latent representation vector. The iteration terminates when the updated latent representation vector satisfies the normal Gaussian mixture component determination condition. S64. Record the perturbation vector that makes the potential representation vector first satisfy the normal Gaussian mixture component determination condition, and calculate the norm of the perturbation vector as the minimum perturbation cost of the corresponding anomalous candidate. S65. Repeat S61 to S64 for all abnormal candidates in the abnormal candidate sequence to form a minimum perturbation cost sequence that corresponds one-to-one with the abnormal candidate sequence.

7. A 3D security protection sensor early warning and protection method based on deep learning according to claim 6, characterized in that, The restricted reverse optimization in S6 specifically includes: The potential representation vector corresponding to the abnormal candidate is used as the initial state vector. The current state vector is set to be equal to the initial state vector, the iteration count is set to zero, and the perturbation vector is initialized to a zero vector with the same dimension as the potential representation vector. While keeping the DAGMM network parameters fixed, the abnormal energy value corresponding to the current state vector is calculated based on the Gaussian mixture model in the DAGMM network, and the difference between the abnormal energy value and the preset abnormal energy value threshold is calculated. Calculate the gradient vector of the abnormal energy value with respect to the current state vector based on the current state vector, and take the opposite direction of the gradient vector as the direction of the perturbation increment vector; The amplitude of the disturbance increment vector is determined based on the difference between the abnormal energy value and the preset abnormal energy value threshold. The amplitude is obtained by multiplying the difference by the preset scaling factor, and amplitude normalization is performed on the disturbance increment vector. The perturbation increment vector, after amplitude determination and normalization, is superimposed on the current state vector to obtain the updated potential representation vector, and the perturbation vector is updated to the vector sum of the historical perturbation vector and the perturbation increment vector. The corresponding abnormal energy value is calculated based on the updated latent representation vector. When the abnormal energy value is less than the preset abnormal energy value threshold, it is determined that the updated latent representation vector satisfies the normal Gaussian mixture component determination condition and the iteration is terminated. When the updated latent representation vector does not meet the normal Gaussian mixture component determination condition, the updated latent representation vector is set as the new current state vector, the iteration count is incremented by one, and the iteration is terminated when the iteration count reaches the preset iteration termination condition.

8. The 3D security protection sensor early warning and protection method based on deep learning according to claim 7, characterized in that, Specifically, S7 includes: S71. Obtain the abnormal energy value sequence and the minimum disturbance cost sequence, and match the abnormal energy value and the minimum disturbance cost one by one according to the index relationship of the abnormal candidates in the time series to form a joint judgment data sequence of abnormal candidates. S72. Based on a pre-set set of early warning thresholds, perform interval judgment processing on the abnormal energy value and minimum disturbance cost in the abnormal candidate joint judgment data sequence, map the abnormal energy value to an energy level identifier, and map the minimum disturbance cost to a reversibility level identifier. S73. Based on the energy level identifier and the reversibility level identifier, execute the joint mapping rule to determine the corresponding early warning level for each anomaly candidate, and associate the early warning level with the time index and spatial unit index number corresponding to the anomaly candidate to generate an early warning event record. S74. Summarize and output the generated early warning event records to form a set of early warning events containing early warning level, time information and spatial location information, which serves as the early warning output result of the 3D safety protection sensor.

Images