A method and system for monitoring and identifying movement of objects within a power scene work area

Abstract

The application provides a kind of object movement monitoring identification method and system in electric power scene operation area, belong to electric power inspection technical field.The method includes: obtaining binocular camera left and right view, detecting abnormal target and extracting feature point pair, generating sparse three-dimensional point cloud by triangulation reconstruction, and then densifying to obtain dense point cloud, fitting three-dimensional bounding box and calculating center coordinates and size;The current frame target is associated with historical frame, and the smoothed position, velocity and size change trend are obtained by Kalman filtering;Dynamic correction is used by segmented distance attenuation function and motion state, and high-risk targets are quantified and marked;The spatial proximity between high-risk targets is calculated, and high-risk target pairs are output;Laser radar point cloud is obtained, and binocular vision recognition results are double matched, real intrusion objects are confirmed by point cloud feature analysis, and fusion information is output.The application can significantly improve the reliability and real-time performance of electric power scene intrusion monitoring.

Description

Technical Field

[0001] This invention belongs to the field of power operation safety assurance technology, and in particular relates to a method and system for monitoring and identifying the movement of objects in a power operation area. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Object movement monitoring and identification technology in power operation and maintenance, infrastructure construction, and other scenarios is an important means to ensure the safety of personnel and facilities in critical power special operation environments such as substations, distribution lines, and transmission lines. As the scale of power systems continues to expand and the operating environment becomes increasingly complex, timely and accurate grasp of the movement information of various objects (personnel, machinery) in various operation areas is of critical significance for preventing risks such as external intrusion and collisions in dangerous areas (e.g., energized areas, deep foundation pits).

[0004] Currently, most power scene monitoring solutions rely on single video image analysis. These solutions deploy cameras to capture on-site footage and use motion detection or target recognition algorithms to initially detect abnormal changes in the scene. However, video images themselves only provide two-dimensional planar information, lacking depth dimension, making it difficult for the system to obtain the precise spatial location and true physical characteristics of intruding objects. When the intruding object is far away, moving at high speed, or partially obscured, the system struggles to determine its actual size, shape, and relative positional relationship with surrounding power equipment, leading to frequent false alarms or missed alarms. Especially in complex scenarios requiring continuous tracking of multiple moving targets, the accuracy of determining the relative movement relationships between objects and the risk of collision drops significantly.

[0005] Video detection can only obtain the pixel coordinates of the intruding object on the image plane, while power scenarios have extremely high requirements for spatial positioning accuracy. For example, when it is necessary to link a PTZ camera for detailed capture or to activate an audible and visual alarm device, relying solely on image coordinates often results in a scanning range that is too large or too small, making it impossible to efficiently locate the actual three-dimensional area where the target is located, thus reducing the efficiency and accuracy of the linked response.

[0006] Furthermore, the motion of an intruding object in real three-dimensional space, including its speed and direction, trajectory trends, and spatial proximity to other objects, directly determines the severity of potential safety risks. However, existing methods struggle to effectively map two-dimensional image features to three-dimensional physical properties, resulting in significant gaps in the analysis of dynamic interactions. For example, when a person carrying tools approaches high-voltage equipment, the system may only detect movement on a plane but fail to promptly determine whether the person has entered a dangerous distance threshold or accurately assess whether a potential collision path has formed with other moving objects. Summary of the Invention

[0007] To overcome the shortcomings of the prior art, the present invention provides a method and system for monitoring and identifying object movement within a power scenario operation area, aiming to solve problems such as missing three-dimensional information, unstable multi-target tracking, and delayed risk warning in the prior art.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a method for monitoring and identifying the movement of objects within a power operation area; A method for monitoring and identifying object movement within a power operation area, comprising: Acquire left and right view images from a binocular camera, detect abnormal targets and extract 2D coordinates and bounding boxes, extract scale-invariant features within the bounding boxes of the left and right views, and obtain high-confidence feature point pairs through bidirectional nearest neighbor matching and ratio test; The disparity value is calculated based on the feature point pairs. Triangulation reconstruction is performed by combining camera intrinsic parameters and baseline length to obtain a sparse 3D point cloud. A 3D dense point cloud is generated based on the sparse point cloud and bounding box constraints. After filtering, a 3D bounding box is fitted and the target center coordinates and physical dimensions are calculated. The target center coordinates, physical dimensions, and bounding box vertex coordinates obtained in the current frame are associated with historical frames, and the smoothed position, velocity, and size trends are obtained through Kalman filtering. Based on the smoothed state, the initial risk value is calculated using the piecewise distance decay function, and then dynamically corrected by the approach rate and time accumulation factor to obtain the comprehensive risk value and mark high-risk targets. Calculate the spatial proximity between high-risk targets and sort and output the results; The system acquires lidar point cloud data, obtains candidate point cloud clusters through ground segmentation and clustering, performs dual matching of position and size with high-risk targets of binocular vision, and outputs fused alarm information after confirming the real intrusion object.

[0009] As a further technical solution, left and right view images from a binocular camera are acquired, abnormal targets are detected, and two-dimensional coordinates and bounding boxes are extracted. Scale-invariant features are extracted within the bounding boxes of the left and right views. High-confidence feature point pairs are obtained through bidirectional nearest neighbor matching and ratio testing, including: The left and right view images of the current frame are acquired from the binocular camera, and the left and right view images are synchronously calibrated to obtain calibrated image pairs; For the calibrated image pair, a pre-trained deep neural network target detector is used to detect abnormal targets, and the two-dimensional pixel coordinates and bounding box coordinates of the abnormal targets are extracted to obtain a coordinate set; Using the two-dimensional bounding box of each anomalous target as the region of interest, SIFT feature descriptors are extracted from the corresponding regions in the left and right views respectively, and feature vector sets are generated. Feature point matching is performed in the left and right views using a bidirectional nearest neighbor matching algorithm. Lowe's ratio test is applied to the preliminary matching results to retain feature point pairs with a matching distance ratio less than a preset threshold, thus obtaining corresponding feature point pairs for the same abnormal target.

[0010] As a further technical solution, disparity values ​​are calculated based on feature point pairs, and triangulation reconstruction is performed in conjunction with camera intrinsic parameters and baseline length to obtain a sparse 3D point cloud. A dense 3D point cloud is then generated based on the sparse point cloud and bounding box constraints, including: For each pair of corresponding feature points, the pixel disparity value is calculated. The disparity value is determined by the difference in the horizontal coordinates of the corresponding points of the same abnormal target in the left and right views, thus obtaining a set of disparity values. Obtain the pre-calibrated camera intrinsic parameter matrix and baseline length. Based on the disparity value set combined with the camera intrinsic parameter matrix and baseline length, perform triangulation calculation through inverse projection transformation to reconstruct the sparse three-dimensional spatial point cloud coordinate set of the abnormal target. Obtain the coordinates of the two-dimensional bounding box of the abnormal target, and use the epipolar geometry constraint of the stereo camera to perform a semi-global block matching algorithm within the bounding boxes of the left and right views to generate an initial disparity map. By combining the sparse three-dimensional point cloud coordinate set, guided filtering and depth value correction are performed on the initial disparity map to obtain an optimized dense disparity map; Back-projection is performed based on the optimized dense disparity map and camera parameters to generate a 3D dense point cloud representation of the anomalous target.

[0011] As a further technical solution, a 3D bounding box is fitted after filtering the 3D dense point cloud, and the target center coordinates and physical dimensions are calculated, including: Set the voxel mesh side length parameter, downsample the 3D dense point cloud, and replace all points in each non-empty voxel with its centroid to obtain the filtered point cloud set; For the filtered point cloud set, calculate the average distance from each point to its K nearest neighbors and the distance distribution of the entire point cloud, remove points whose distance mean exceeds the global mean by M standard deviations, and obtain the denoised point cloud set. Based on the spatial distribution of points in the denoised point cloud set, calculate the minimum and maximum values ​​of the three-dimensional coordinates of all points, and construct an axis-aligned bounding box with the extreme values ​​of the denoised point cloud coordinates as the boundary to obtain the bounding box of the intrusion target. Calculate the arithmetic mean of the coordinates of the eight vertices of the bounding box to obtain the center coordinates of the bounding box in the world coordinate system; The physical dimensions of the bounding box of the intrusion target are calculated based on the difference between the extreme coordinate values.

[0012] As a further technical solution, the target center coordinates, physical dimensions, and bounding box vertex coordinates obtained in the current frame are correlated with historical frames, and smoothed position, velocity, and size trends are obtained through Kalman filtering, including: Obtain the target's 3D center coordinates, physical dimensions, and the coordinates of the eight vertices of its bounding box from the current frame; Calculate the three-dimensional Mahalanobis distance and the cosine similarity of the length, width and height dimensions between the current frame target and the previous frame states of all historical tracked targets; Target pairs whose Mahalanobis distance is less than a threshold and whose cosine similarity is greater than a threshold are selected as candidate matching pairs; For the candidate matching pairs, the Hungarian algorithm is used for optimal matching to obtain a stable associated target ID mapping relationship; for successfully associated targets, the current frame observation value and the Kalman filter prediction value are fused and updated. The smoothed three-dimensional position and velocity state estimates are obtained by updating the equations using Kalman filtering. The first-order difference is calculated based on the size state after smoothing multiple consecutive frames to obtain the size change trend vector.

[0013] As a further technical solution, based on the smoothed state, a preliminary risk value is calculated using a piecewise distance decay function, and then dynamically corrected using approach rate and time accumulation factors to obtain a comprehensive risk value and mark high-risk targets, including: The risk values ​​of the target to the hazardous area and equipment are calculated using a piecewise distance attenuation function, and a preliminary risk value is obtained by weighted fusion. The risk increment of the target approach rate and the cumulative risk increment over time are calculated and then summed to obtain the comprehensive risk value. When the overall risk value exceeds the static threshold or approaches the rate exceeding the dynamic threshold, it is marked as a high-risk target.

[0014] As a further technical solution, the spatial proximity between high-risk targets is calculated and sorted, including: Calculate the target's current 3D Euclidean distance, minimum distance to the future predicted trajectory, and bounding box volume overlap. The three indicators are weighted and fused to obtain a proximity value. The values ​​are sorted from largest to smallest and high-risk target pairs are output.

[0015] A second aspect of the present invention provides a system for monitoring and identifying the movement of objects within a power scenario operating area.

[0016] A system for monitoring and recognizing object movement within a power operation area, comprising: The data processing module is configured to: acquire left and right view images from a binocular camera, detect abnormal targets and extract two-dimensional coordinates and bounding boxes, extract scale-invariant features within the bounding boxes of the left and right views, and obtain high-confidence feature point pairs through bidirectional nearest neighbor matching and ratio testing; The bounding box processing module is configured to: calculate the disparity value based on the feature point pairs, perform triangulation reconstruction by combining the camera intrinsic parameters and the baseline length to obtain a sparse 3D point cloud, generate a 3D dense point cloud based on the sparse point cloud and the bounding box constraint, fit a 3D bounding box after filtering, and calculate the target center coordinates and physical size. The calculation and processing module is configured to associate the target center coordinates, physical size and bounding box vertex coordinates obtained in the current frame with historical frames, and obtain the smoothed position, velocity and size trends through Kalman filtering. The marking module is configured to: calculate the initial risk value based on the smoothed state using the piecewise distance decay function, dynamically correct the comprehensive risk value by combining the approach rate and time accumulation factor, and mark high-risk targets. The trajectory tracking module is configured to calculate the spatial proximity between high-risk targets and output the sorted results. The secondary identification module is configured to: acquire lidar point cloud data, obtain candidate point cloud clusters through ground segmentation and clustering, perform dual matching of position and size with high-risk targets of binocular vision, and output fused alarm information after confirming the real intrusion object.

[0017] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a method for monitoring and identifying object movement within a power scenario operating area as described in the first aspect of the present invention.

[0018] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the object movement monitoring and identification method in a power scene operation area as described in the first aspect of the present invention.

[0019] The above one or more technical solutions have the following beneficial effects: (1) This invention, through binocular visual feature matching, triangulation 3D reconstruction, and dense point cloud optimization, can directly obtain the 3D coordinates, physical size, and spatial contour of a target in the world coordinate system, fundamentally solving the problems of traditional 2D images lacking depth information and being susceptible to interference from illumination and occlusion. The system can accurately distinguish between small targets at a distance and large targets at a close distance, stably calculate the actual distance between the target and power equipment, and significantly improve positioning and measurement accuracy. Combined with voxel filtering and outlier removal processing, the point cloud data is clean and regular, and the 3D bounding box fitting is stable and reliable. It can continuously output high-precision spatial information in complex power scenarios such as substations and transmission corridors, providing solid data support for risk assessment.

[0020] (2) This invention correlates the three-dimensional center coordinates, physical dimensions, and bounding box vertex information of the target in the current frame with the spatial distance and size similarity of the tracked targets in historical frames. It employs a dual judgment of Mahalanobis distance and cosine similarity, combined with the Hungarian algorithm to achieve optimal matching, and updates the state vector containing position, velocity, and size through Kalman filtering. This mechanism can effectively cope with complex situations such as sudden changes in illumination, partial occlusion, and rapid target movement, significantly reducing identity jumps, drifts, and loss phenomena during target tracking. In power scenarios with dense equipment and severe interference, the system can still maintain stable cross-frame tracking of multiple targets, ensuring the continuity and reliability of risk identification.

[0021] (3) This invention uses a piecewise distance attenuation function to calculate the risk value of the target to the preset danger zone and the surface of fixed equipment, and performs normalized weighted fusion to obtain a preliminary risk quantification value. On this basis, the approach rate risk increment and the time cumulative risk increment are further introduced to dynamically correct the preliminary risk value: the approach rate increment quantifies the urgency of the target "rushing" towards the danger point, and the time cumulative increment reflects the cumulative effect of the risk duration. The comprehensive risk value combined with static and dynamic dual threshold judgment can simultaneously capture sudden high-speed approach events and long-term chronic risk accumulation, making the risk warning closer to the actual movement intention of the target and greatly reducing the probability of missed alarms and false triggers.

[0022] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0023] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0024] Figure 1 This is a flowchart of the method in the first embodiment.

[0025] Figure 2This is a system structure diagram of the second embodiment. Detailed Implementation

[0026] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0027] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0028] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0029] The overall approach of this invention addresses the issues of missing 3D information, unstable tracking, and delayed risk warnings in multi-target intrusion monitoring in power scenarios. First, it acquires left and right views using a binocular camera, detects abnormal targets, and extracts high-confidence feature point pairs. These are then triangulated, reconstructed, and densified to generate a dense 3D point cloud of the target, which is then fitted with an accurate 3D bounding box. Next, Kalman filtering is used to achieve stable cross-frame tracking and state smoothing for multiple targets. Based on this, a piecewise distance attenuation function and dynamic motion state correction are combined to quantify risk and mark high-risk targets. Finally, a dual matching and identification of spatial location and geometric dimensions using lidar point clouds is performed to confirm the actual intruding object and output fused information, achieving accurate perception, stable tracking, and intelligent early warning for multiple targets in power scenarios.

[0030] Example 1 This embodiment discloses a method for monitoring and identifying object movement within a power scenario operating area. It obtains accurate spatial information of the target through binocular vision 3D reconstruction, achieves stable multi-target tracking by combining Kalman filtering, completes high-risk judgment by using a dynamic risk model, and effectively reduces false alarms by relying on secondary identification by lidar, thus comprehensively improving the reliability and real-time performance of security monitoring in power scenarios.

[0031] like Figure 1 As shown, a method for monitoring and identifying object movement within a power scenario operating area includes: Step S1: Acquire left and right view images from the binocular camera, detect abnormal targets and extract two-dimensional coordinates and bounding boxes, extract scale-invariant features within the bounding boxes of the left and right views, and obtain high-confidence feature point pairs through bidirectional nearest neighbor matching and ratio test.

[0032] Step S11: Acquire the left and right view images of the current frame from the binocular camera, perform synchronous calibration on the left and right view images to obtain calibrated image pairs; for the calibrated image pairs, use a pre-trained deep neural network target detector to detect abnormal targets, extract the two-dimensional pixel coordinates and bounding box coordinates of the abnormal targets to obtain a coordinate set; Step S12: Using the two-dimensional bounding box of each abnormal target as the region of interest, extract SIFT feature descriptors in the corresponding regions of the left and right views respectively, and generate feature vector groups. Step S13: Feature point matching is performed in the left and right views using a bidirectional nearest neighbor matching algorithm. Lowe's ratio test is applied to the preliminary matching results to retain feature point pairs whose matching distance ratio is less than a preset threshold, thereby obtaining corresponding feature point pairs for the same abnormal target. Disparity values ​​are calculated for the corresponding feature point pairs, and the disparity values ​​are determined by the pixel position difference of corresponding points in the left and right views to obtain depth information.

[0033] It should be noted that, in this embodiment, feature point matching is performed in the left and right views using a bidirectional nearest neighbor matching algorithm. First, the distance between the SIFT feature descriptors of the left and right views is calculated. Then, bidirectional nearest neighbor matching is performed. Specifically, the distance between the SIFT feature descriptors of the left and right views is calculated using Euclidean distance:

[0034] in, SIFT feature descriptor for the left view; This is the SIFT feature descriptor for the right view.

[0035] Then, for each feature point in the left view Find the nearest neighbor in the right view (minimum distance) and next nearest neighbors (distance Next, a reverse check is performed, checking each matching point in the right view. In the left view, nearest neighbor matching is also performed; if the best match is still... If so, then retain it; Furthermore, through the Lowe's ratio test processing described in the above embodiments, i.e. ,in A preset distance ratio threshold, typically between 0.7 and 0.8, is used to obtain corresponding feature point pairs for the same anomalous target. .

[0036] By extracting SIFT features within the target detection bounding box and employing bidirectional matching and ratio testing, feature matching is limited to the effective target area and ambiguous matches are eliminated. This area limitation reduces background interference and computational load, bidirectional matching ensures the uniqueness of correspondences, and ratio testing filters out fuzzy matching points, thereby significantly improving the matching accuracy and robustness of feature point pairs between the left and right views. This provides a clean and reliable data input source for subsequent high-precision disparity calculation and 3D reconstruction, serving as a fundamental guarantee for the accuracy of the entire 3D perception process.

[0037] Step S2: Calculate the disparity value based on the feature point pairs, and perform triangulation reconstruction by combining the camera intrinsic parameters and baseline length to obtain a sparse 3D point cloud. Generate a 3D dense point cloud based on the sparse point cloud and bounding box constraints. After filtering, fit a 3D bounding box and calculate the target center coordinates and physical dimensions.

[0038] Step S21: Calculate the pixel disparity value for each pair of corresponding feature points, and determine the disparity value by the difference in the horizontal coordinates of the corresponding points of the same abnormal target in the left and right views, so as to obtain a set of disparity values. Step S22: Obtain the pre-calibrated camera intrinsic parameter matrix and baseline length. Based on the disparity value set combined with the camera intrinsic parameter matrix and baseline length, perform triangulation calculation through inverse projection transformation to reconstruct the sparse three-dimensional spatial point cloud coordinate set of the abnormal target. It should be noted that, in the above embodiments, during the specific processing, based on the intrinsic parameter matrix of the left and right cameras being K, the baseline length being B, and the pixel coordinates of a certain feature point in the left and right views being respectively... and parallax ; Perform parallax-to-depth conversion:

[0039] in, For camera focal length, This represents the depth value of the target point.

[0040] Next, triangulation is performed using inverse projection transformation, where the camera coordinate system is:

[0041]

[0042]

[0043] in,( , () represents the coordinates of the principal point in the image. From this, we obtain the sparse 3D point cloud coordinate set of the abnormal target, i.e. .

[0044] Step S23: Obtain the coordinates of the two-dimensional bounding box of the abnormal target, and use the epipolar geometric constraints of the binocular camera to perform a semi-global block matching algorithm within the bounding boxes of the left and right views to generate an initial disparity map. The semi-global block matching algorithm calculates the optimal disparity value for each pixel within the same target bounding box region in both left and right views, generating a dense disparity map. First, it calculates the matching cost (using Census transform as an example). For each pixel, it compares the grayscale values ​​of its neighboring pixels with the center pixel, generating a bit string.

[0045] Where is the Census transform bit string of pixel p; This is a bit string concatenation operation; For comparison functions, These are the gray values ​​of the neighboring pixel q and the center pixel p, respectively.

[0046] The cost function is the Hamming distance between the Census strings of the left and right graphs, i.e.:

[0047] in, Let be the matching cost of pixel p when the disparity is d; Let Hamming distance function be used. The Census transform bit string of pixel p in the left view; Candidate matching pixel p for the right view The Census transform bit string of d.

[0048] The semi-global block matching algorithm (SGM) does not simply take the disparity with the minimum cost, but instead incorporates a smoothing term constraint and defines an energy function:

[0049] in, This represents the total energy value corresponding to disparity map D; the first term of the formula is the matching cost term. The first term is the disparity value assigned to pixel p; the second term is a penalty term for small disparity changes. The first term is the penalty coefficient for small parallax changes; the third term is the penalty for large parallax changes.

[0050] Then, multi-path aggregation is performed, i.e., one-dimensional dynamic programming is performed along the r direction:

[0051] in, This is for smoothing out penalty terms.

[0052] Finally, sum the costs from all directions:

[0053] Take each pixel smallest As the initial disparity, an initial disparity map is generated based on all the initial disparities. The initial disparity map is input into step S24, where it is combined with sparse point cloud for guided filtering and depth correction to eliminate incorrect matching caused by texture duplication, occlusion, etc.

[0054] Step S24: Combine the sparse three-dimensional spatial point cloud coordinate set, perform guided filtering and depth value correction on the initial disparity map to obtain an optimized dense disparity map; perform back projection based on the optimized dense disparity map and camera parameters to generate a three-dimensional dense point cloud representation of the abnormal target.

[0055] The above steps combine sparse point cloud and semi-global block matching for dense disparity estimation and optimization, achieving 3D reconstruction from feature points to the complete target surface. Specifically, the sparse point cloud provides reliable depth priors and constraints for block matching. Guided filtering utilizes these priors to smooth the disparity map while preserving edges, and the correction step eliminates inconsistencies, ultimately generating a high-quality dense point cloud. This method overcomes the shortcomings of pure feature matching, which can only obtain sparse contours, and pure dense matching, which is prone to failure in weakly textured regions. It can generate a more complete and less noisy 3D representation of the target surface, providing crucial data support for subsequent accurate calculation of the target's physical size and occupied space.

[0056] Furthermore, in step S2, voxel filtering and outlier removal are performed on the 3D dense point cloud. An axis-aligned bounding box fitting method is used to determine the 3D bounding box of the intrusion target. The center coordinates and physical dimensions (length, width, and height) of the bounding box in the world coordinate system are calculated, including: Step S25: Set the voxel mesh side length parameter V, downsample the 3D dense point cloud, and replace all points in each non-empty voxel with its centroid to obtain the filtered point cloud set. Step S26: For the filtered point cloud set, calculate the average distance from each point to its K nearest neighbors and the distance distribution of the entire point cloud, remove points whose distance mean exceeds the global mean by M standard deviations, and obtain the denoised point cloud set. Step S27: Based on the spatial distribution of points in the denoised point cloud set, calculate the minimum and maximum values ​​of the three-dimensional coordinates of all points, so as to... Construct an axis-aligned bounding box for the boundary to obtain the bounding box of the intrusion target; Step S28: Calculate the arithmetic mean of the coordinates of the eight vertices of the bounding box to obtain the center coordinates of the bounding box in the world coordinate system; calculate based on the boundary coordinates... This allows us to obtain the physical dimensions of the bounding box of the intrusion target: length, width, and height.

[0057] 3D dense point clouds are normalized and cleaned through parameterized voxel downsampling and outlier removal based on statistical distribution. Specifically, voxel downsampling significantly reduces data volume and noise while preserving the point cloud shape, and the outlier removal method based on K-nearest neighbor distance statistics can adaptively remove discrete noise points far from the main point cloud. The processed point cloud data is cleaner and more uniformly distributed, allowing the axis-aligned bounding box constructed directly from coordinate extrema to more tightly and stably wrap the target. The calculated center coordinates and length, width, and height dimensions are less affected by noise, resulting in more reliable and consistent measurement results and improved accuracy in 3D positioning and dimensional determination.

[0058] Step S3: Associate the target center coordinates, physical dimensions, and bounding box vertex coordinates obtained in the current frame with historical frames, and obtain the smoothed position, velocity, and size trends through Kalman filtering.

[0059] Step S31: Obtain the target's three-dimensional center coordinates, physical dimensions, and the coordinates of the eight vertices of the bounding box from the current frame. Calculate the three-dimensional Mahalanobis distance and the cosine similarity of the length, width, and height dimensions between the target in the current frame and the previous frame states of all historical tracked targets.

[0060] The target observation vector detected in the current frame is (3D center coordinates), the mean of the predicted state of the historical tracked target in the previous frame is The predicted covariance matrix is ​​S. The Mahalanobis distance is calculated as follows:

[0061] in, This is the Mahalanobis distance between the current frame's target observations and the historical target predictions. This Mahalanobis distance considers not only coordinate differences but also the uncertainty (covariance) of the Kalman filter estimation, making it more robust than the Euclidean distance. It allows for larger positional deviations when the prediction covariance is large.

[0062] The calculation of cosine similarity of corals is based on the size vector of the target. (Length, width, and height).

[0063] Current frame target size Historical target size : The range of values ​​is [ 1,1][ [1,1], the larger the value, the more consistent the size and shape; The two indicators mentioned above are used together to construct candidate matching pairs: only those that simultaneously satisfy... And cos_sim Only the target pairs are considered for optimal matching in the Hungarian algorithm. This avoids mismatches that rely solely on position or size.

[0064] Step S32, select Mahalanobis distances less than the threshold. And the cosine similarity is greater than the threshold. The target pair is used as a candidate matching pair.

[0065] Step S33: For the candidate matching pairs, the Hungarian algorithm is used for optimal matching to obtain a stable associated target ID mapping relationship; for successfully associated targets, their current frame observation values ​​(center coordinates, length, width, and height) are fused and updated with the Kalman filter's predicted values; the state vector of the Kalman filter is defined as... , representing the three-dimensional position, velocity, and bounding box size, respectively.

[0066] Specifically, based on the candidate matching pairs obtained in step S32, an M×N cost matrix is ​​constructed, where M is the number of targets detected in the current frame and N is the number of historically tracked targets. The matrix elements represent the combined cost of Mahalanobis distance and cosine similarity. The Hungarian algorithm is run on the cost matrix to find the globally optimal matching scheme that minimizes the total cost, ensuring that each target matches at most one historical trajectory, thus obtaining the target ID mapping relationship. Unmatched detected targets are initialized as new trajectories, and unmatched historical trajectories are temporarily suspended for several frames before being deleted.

[0067] For successfully associated targets, the current frame observations are fused with the Kalman filter predictions. First, the state vector is expanded, defining the state vector for each target as:

[0068] in,( ( ) is the center coordinate, ( ) represents speed, ( ( ) represents the dimensions of the enclosing box.

[0069] Then, based on the state and covariance of the previous frame, the state of the current frame is predicted according to the uniform motion model:

[0070] in, According to the kth The state vector of the k-th frame predicted from the estimate of the first frame; Here is the state transition matrix. Here is the covariance matrix of the predicted state; Let be the process noise covariance.

[0071] Then, an update step is performed. For successfully associated targets, their current frame observation vector is:

[0072] in, The coordinates of the three-dimensional center of the target detected in the previous frame; This represents the size of the bounding box of the target detected in the current frame. (Observation vector) Since it contains no velocity component, the velocity cannot be directly observed and can only be obtained indirectly through state estimation using a Kalman filter.

[0073] Finally, update the parameters according to the following update formula:

[0074] in, For residuals; For information covariance; Kalman gain; Update to the optimal state; Update for optimal covariance; To observe the noise covariance; It is an identity matrix.

[0075] Finally, the output shows the smoothed position, velocity, size, and size change trend.

[0076] The smoothed three-dimensional position and velocity state estimates are obtained by updating the equations using Kalman filtering. Based on the size state after smoothing multiple consecutive frames Calculate its first-order difference to obtain the size change trend vector.

[0077] By fusing Mahalanobis distance, cosine similarity, and the Hungarian algorithm for optimal matching, stable multi-target cross-frame tracking in complex scenes is achieved. In practice, Mahalanobis distance considers the uncertainty of state estimation and is more robust than Euclidean distance; cosine similarity effectively assesses the continuity of size and shape; the combination of both improves the accuracy of association discrimination. The Hungarian algorithm ensures globally optimal matching. The extended Kalman state vector (including size) enables integrated smooth estimation and prediction of target position, velocity, and shape.

[0078] Step S4: Based on the smoothed state, calculate the preliminary risk value using the piecewise distance decay function, and dynamically correct it by combining the approach rate and time accumulation factor to obtain the comprehensive risk value and mark high-risk targets.

[0079] Step S41: Based on the smoothed three-dimensional position, velocity and size change trends, the expected three-dimensional position of the target in the next frame and its covariance uncertainty region are obtained using the prediction step of the Kalman filter.

[0080] Step S42, define the piecewise distance decay function :

[0081] in, To ensure continuity; d is the shortest Euclidean distance from the target to the hazardous area or equipment surface. To maintain an absolutely safe distance, Let k1, k2, and λ be the warning distance, and k1, k2, and λ be constant coefficients. This is the base value for maximum risk.

[0082] Step S43: Calculate the risk value of the target to the i-th preset danger zone. and the risk value to the surface of the j-th fixed equipment. ; For hazardous areas (such as electrified areas or deep foundation pits), the distance from a target within a spatial range to that area is usually defined. This represents the shortest Euclidean distance from the nearest point in the target bounding box to the region boundary. For a fixed device surface, the device is modeled as a series of triangular patches or bounding boxes. Distance This represents the shortest distance from the target to the device surface. Then, it is substituted into the piecewise distance attenuation function described above. This yields the risk value corresponding to each risk source.

[0083] Step S44: Perform fusion using a weighted summation method:

[0084] in, , These are the preset normalized weighting coefficients. This is the preliminary risk value.

[0085] Step S45, dynamically correct the preliminary risk quantification value based on the target's motion state, including: Obtain the preliminary risk quantification value ; Calculate the approach rate risk increment of the target: Let the target's velocity vector be V, and the vector from the target to the nearest risk point P be D. Then the approach speed... , where θ is the angle between V and D; Risk increment Where γ is the increment coefficient, Approaching the rate threshold; Calculate the cumulative risk increment over time: Assume the target is continuously in a risk state. If the number of frames is N, then the cumulative risk increment over time is... , where η is the maximum cumulative increment and μ is the attenuation coefficient; Step S46, calculate the corrected comprehensive risk value: ; Assess the risk status: If If the target is marked as a high-risk target, then the target is marked as such; This is a static risk threshold. This is the dynamic approach rate threshold.

[0086] Step S5: Calculate the spatial proximity between high-risk targets and sort and output the results.

[0087] Obtain the 3D center coordinates of high-risk target pair A and B in the current frame. , Calculate its Euclidean distance =|| - ||; Obtain the predicted state sequences of targets A and B over the next T seconds using a Kalman filter, and calculate the minimum Euclidean distance between all pairs of points on the predicted trajectories of the two targets. ; Calculate the overlap of the bounding box volume occupancy of targets: Let the bounding box volumes of targets A and B be respectively... Its bounding box union volume is Then the degree of overlap Then calculate the final proximity value. :

[0088] in , , These are preset weighting coefficients, and α + β + χ = 1. According to the... Sort all high-risk target pairs by value from largest to smallest and output the sorting results.

[0089] By fusing three indicators—current distance, predicted minimum distance, and volumetric overlap—the spatial proximity between high-risk target pairs is calculated. Specifically, current distance reflects immediate risk, predicted minimum distance warns of potential future conflicts, and volumetric overlap assesses the degree of overlap in spatial occupancy. Through weighted fusion, Numerical synthesis characterizes the overall proximity and conflict probability of target pairs in space and time, providing a more comprehensive picture than a single distance indicator.

[0090] Furthermore, the dynamic interaction relationships between multiple targets due to spatial proximity and movement trends are recorded for subsequent coordinated response processes, including: Obtain the sorted high-risk target pairs and their proximity values. ; Define an interaction graph G=(V, E), where vertices V represent high-risk targets, edges E represent the interaction relationships between target pairs, and the weights of the edges are... Based on the interaction graph G, a community detection algorithm is used to identify strongly associated target communities. For each target community, calculate its overall risk level:

[0091] Among them, | | represents the number of edges within the community; based on the overall risk level. Based on the movement trends of targets within the community (such as average velocity direction, diffusion or aggregation), a sequence of coordinated handling instructions with different priorities is generated, and this sequence, along with the corresponding target spatial bounding box information, is output to the downstream handling system.

[0092] Analysis of the above technical solution reveals that this step, by constructing a weighted interaction graph and applying a community detection algorithm, upgrades the analysis of discrete high-risk target pairs to the identification of the overall risk of target groups. During the execution process, graph construction using proximity as edge weights formally describes the complex interaction network between multiple targets; the community detection algorithm automatically identifies groups (communities) with tight internal connections and sparse external connections. The overall risk level of the community is calculated, integrating the intensity of intra-group interactions with individual risk levels.

[0093] Step S6: Acquire lidar point cloud data, obtain candidate point cloud clusters through ground segmentation and clustering, perform dual matching of position and size with high-risk targets of binocular vision, and output fused alarm information after confirming the real intrusion object.

[0094] Step S61: Calculate the Euclidean distance between the center coordinates of the lidar point cloud cluster and the center coordinates of the binocular vision high-risk target bounding box. If the distance is less than the preset spatial distance threshold, it is determined to be a preliminary position match. Step S62: For the target with preliminary position matching, calculate the intersection-union ratio (IoU) of the length, width, and height of its lidar point cloud cluster and the length, width, and height of its binocular vision bounding box. If the IoU is greater than the preset size matching threshold, it is determined to be a geometric size matching. Step S63: For a target that simultaneously meets the requirements of spatial location and geometric size matching, analyze the reflection intensity distribution and geometric shape characteristics of its lidar point cloud cluster. If the reflection intensity is within the preset range of common intruders and the compactness of the point cloud cluster is good, then it is confirmed as a real intruder. Step S64: For targets that are only detected by lidar but not identified as high-risk by binocular vision, but whose point cloud features match the features of real intruding objects, they are additionally marked as high-risk targets.

[0095] Example 2 This embodiment discloses an object movement monitoring and recognition system within a power scenario operation area; like Figure 2 As shown, a system for monitoring and recognizing object movement within a power scenario work area includes: Data processing module 10 is configured to: acquire left and right view images from a binocular camera, detect abnormal targets and extract two-dimensional coordinates and bounding boxes, extract scale-invariant features within the bounding boxes of the left and right views, and obtain high-confidence feature point pairs through bidirectional nearest neighbor matching and ratio testing. The bounding box processing module 20 is configured to: calculate the disparity value based on the feature point pairs, perform triangulation reconstruction by combining the camera intrinsic parameters and baseline length to obtain a sparse 3D point cloud, generate a 3D dense point cloud based on the sparse point cloud and bounding box constraints, fit a 3D bounding box after filtering, and calculate the target center coordinates and physical size. The calculation and processing module 30 is configured to associate the target center coordinates, physical size and bounding box vertex coordinates obtained in the current frame with the historical frames, and obtain the smoothed position, velocity and size trends through Kalman filtering. The marking module 40 is configured to: calculate a preliminary risk value based on the smoothed state using a piecewise distance decay function, dynamically correct the comprehensive risk value by combining the approach rate and time accumulation factor, and mark high-risk targets. The trajectory tracking module 50 is configured to calculate the spatial proximity between high-risk targets and output the sorted results. The secondary identification module 60 is configured to: acquire lidar point cloud data, obtain candidate point cloud clusters through ground segmentation and clustering, perform dual matching of position and size with high-risk targets of binocular vision, and output fused alarm information after confirming the real intrusion object.

[0096] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0097] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for monitoring and identifying object movement within a power scenario operating area as described in Embodiment 1.

[0098] Example 4 The purpose of this embodiment is to provide an electronic device.

[0099] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the object movement monitoring and identification method in a power scene operation area as described in Embodiment 1.

[0100] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0101] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0102] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for monitoring and identifying object movement within a power scenario operating area, characterized in that, include: Acquire left and right view images from a binocular camera, detect abnormal targets and extract 2D coordinates and bounding boxes, extract scale-invariant features within the bounding boxes of the left and right views, and obtain high-confidence feature point pairs through bidirectional nearest neighbor matching and ratio test. The disparity value is calculated based on the feature point pairs. Triangulation reconstruction is performed by combining camera intrinsic parameters and baseline length to obtain a sparse 3D point cloud. A 3D dense point cloud is generated based on the sparse point cloud and bounding box constraints. After filtering, a 3D bounding box is fitted and the target center coordinates and physical dimensions are calculated. The target center coordinates, physical dimensions, and bounding box vertex coordinates obtained in the current frame are associated with historical frames, and the smoothed position, velocity, and size trends are obtained through Kalman filtering. Based on the smoothed state, the initial risk value is calculated using the piecewise distance decay function, and then dynamically corrected by the approach rate and time accumulation factor to obtain the comprehensive risk value and mark high-risk targets. Calculate the spatial proximity between high-risk targets and sort and output the results; The system acquires LiDAR point cloud data, obtains candidate point cloud clusters through ground segmentation and clustering, performs dual matching of position and size with high-risk targets from binocular vision, and outputs fused alarm information after confirming the real intrusion object.

2. The method for monitoring and identifying object movement within a power scenario operating area as described in claim 1, characterized in that, Acquire left and right view images from a stereo camera, detect abnormal targets and extract 2D coordinates and bounding boxes. Extract scale-invariant features within the bounding boxes of the left and right views. Obtain high-confidence feature point pairs through bidirectional nearest neighbor matching and ratio testing, including: The left and right view images of the current frame are acquired from the binocular camera, and the left and right view images are synchronously calibrated to obtain calibrated image pairs; For the calibrated image pair, a pre-trained deep neural network target detector is used to detect abnormal targets, and the two-dimensional pixel coordinates and bounding box coordinates of the abnormal targets are extracted to obtain a coordinate set; Using the two-dimensional bounding box of each anomalous target as the region of interest, SIFT feature descriptors are extracted from the corresponding regions in the left and right views respectively, and feature vector sets are generated. Feature point matching is performed in the left and right views using a bidirectional nearest neighbor matching algorithm. Lowe's ratio test is applied to the preliminary matching results to retain feature point pairs with a matching distance ratio less than a preset threshold, thus obtaining corresponding feature point pairs for the same abnormal target.

3. The method for monitoring and identifying object movement within a power scenario operating area as described in claim 1, characterized in that, Disparity values ​​are calculated based on feature point pairs, and triangulation reconstruction is performed using camera intrinsic parameters and baseline length to obtain a sparse 3D point cloud. A dense 3D point cloud is then generated based on the sparse point cloud and bounding box constraints, including: For each pair of corresponding feature points, the pixel disparity value is calculated. The disparity value is determined by the difference in the horizontal coordinates of the corresponding points of the same abnormal target in the left and right views, thus obtaining a set of disparity values. Obtain the pre-calibrated camera intrinsic parameter matrix and baseline length. Based on the disparity value set combined with the camera intrinsic parameter matrix and baseline length, perform triangulation calculation through inverse projection transformation to reconstruct the sparse three-dimensional spatial point cloud coordinate set of the abnormal target. Obtain the coordinates of the two-dimensional bounding box of the abnormal target, and use the epipolar geometry constraint of the stereo camera to perform a semi-global block matching algorithm within the bounding boxes of the left and right views to generate an initial disparity map. By combining the sparse three-dimensional point cloud coordinate set, guided filtering and depth value correction are performed on the initial disparity map to obtain an optimized dense disparity map; Back-projection is performed based on the optimized dense disparity map and camera parameters to generate a 3D dense point cloud representation of the anomalous target.

4. The method for monitoring and identifying object movement within a power scenario operating area as described in claim 1, characterized in that, After filtering the 3D dense point cloud, a 3D bounding box is fitted, and the target center coordinates and physical dimensions are calculated, including: Set the voxel mesh side length parameter, downsample the 3D dense point cloud, and replace all points in each non-empty voxel with its centroid to obtain the filtered point cloud set. For the filtered point cloud set, calculate the average distance from each point to its K nearest neighbors and the distance distribution of the entire point cloud, remove points whose distance mean exceeds the global mean by M standard deviations, and obtain the denoised point cloud set. Based on the spatial distribution of points in the denoised point cloud set, calculate the minimum and maximum values ​​of the three-dimensional coordinates of all points, and construct an axis-aligned bounding box with the extreme values ​​of the denoised point cloud coordinates as the boundary to obtain the bounding box of the intrusion target. Calculate the arithmetic mean of the coordinates of the eight vertices of the bounding box to obtain the center coordinates of the bounding box in the world coordinate system; The physical dimensions of the bounding box of the intrusion target are calculated based on the difference between the extreme coordinate values.

5. The method for monitoring and identifying object movement within a power scenario operating area as described in claim 1, characterized in that, By associating the target center coordinates, physical dimensions, and bounding box vertex coordinates obtained in the current frame with historical frames, and using Kalman filtering, smoothed position, velocity, and size trends are obtained, including: Obtain the target's 3D center coordinates, physical dimensions, and the coordinates of the eight vertices of its bounding box from the current frame; Calculate the three-dimensional Mahalanobis distance and the cosine similarity of the length, width and height dimensions between the current frame target and the previous frame states of all historical tracked targets; Target pairs whose Mahalanobis distance is less than a threshold and whose cosine similarity is greater than a threshold are selected as candidate matching pairs; For the candidate matching pairs, the Hungarian algorithm is used for optimal matching to obtain a stable associated target ID mapping relationship; for successfully associated targets, the current frame observation value and the Kalman filter prediction value are fused and updated. The smoothed three-dimensional position and velocity state estimates are obtained by updating the equations using Kalman filtering. The first-order difference is calculated based on the size state after smoothing multiple consecutive frames to obtain the size change trend vector.

6. The method for monitoring and identifying object movement within a power scenario operating area as described in claim 1, characterized in that, Based on the smoothed state, a preliminary risk value is calculated using a piecewise distance decay function. This is then dynamically corrected using approach rate and time accumulation factors to obtain a comprehensive risk value, and high-risk targets are marked, including: The risk values ​​of the target to the hazardous area and equipment are calculated using a piecewise distance attenuation function, and a preliminary risk value is obtained by weighted fusion. The risk increment of the target approach rate and the cumulative risk increment over time are calculated and then summed to obtain the comprehensive risk value. When the overall risk value exceeds the static threshold or approaches the rate exceeding the dynamic threshold, it is marked as a high-risk target.

7. The method for monitoring and identifying object movement within a power scenario operating area as described in claim 1, characterized in that, Calculate the spatial proximity between high-risk targets and sort the output, including: Calculate the target's current 3D Euclidean distance, minimum distance to the future predicted trajectory, and bounding box volume overlap. The three indicators are weighted and fused to obtain a proximity value. The values ​​are sorted from largest to smallest and high-risk target pairs are output.

8. A system for monitoring and identifying object movement within a power scenario work area, characterized in that, include: The data processing module is configured to: acquire left and right view images from a binocular camera, detect abnormal targets and extract two-dimensional coordinates and bounding boxes, extract scale-invariant features within the bounding boxes of the left and right views, and obtain high-confidence feature point pairs through bidirectional nearest neighbor matching and ratio testing; The bounding box processing module is configured to: calculate the disparity value based on the feature point pairs, perform triangulation reconstruction by combining the camera intrinsic parameters and the baseline length to obtain a sparse 3D point cloud, generate a 3D dense point cloud based on the sparse point cloud and the bounding box constraint, fit a 3D bounding box after filtering, and calculate the target center coordinates and physical size. The calculation and processing module is configured to associate the target center coordinates, physical size and bounding box vertex coordinates obtained in the current frame with historical frames, and obtain the smoothed position, velocity and size trends through Kalman filtering. The marking module is configured to: calculate the initial risk value based on the smoothed state using the piecewise distance decay function, dynamically correct the comprehensive risk value by combining the approach rate and time accumulation factor, and mark high-risk targets. The trajectory tracking module is configured to calculate the spatial proximity between high-risk targets and output the sorted results. The secondary identification module is configured to: acquire lidar point cloud data, obtain candidate point cloud clusters through ground segmentation and clustering, perform dual matching of position and size with high-risk targets of binocular vision, and output fused alarm information after confirming the real intrusion object.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by the processor, the program implements the steps of the object movement monitoring and identification method in a power scene operation area as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the object movement monitoring and identification method in a power scene operation area as described in any one of claims 1-7.

Images