A real-time visual monitoring method based on a near electric area virtual fence

Abstract

The application discloses a kind of real-time visualization monitoring methods based on near electric area virtual fence, obtains monitoring range instruction;According to the current real-time position of monitoring equipment, ray projection is carried out in 3DGS scene, and the reference monitoring surface is obtained in combination with initial attitude parameter and monitoring range, and the extended early warning area is obtained by unidirectional extension along monitoring direction, and the monitoring area is demarcated in combination with laser radar effective monitoring range;Laser radar monitoring data in monitoring area is collected, object point cloud data is solved to obtain object coordinates, if object coordinates are in extended early warning area, early warning, if in reference monitoring surface, it is determined that object is in reference monitoring surface coordinate;The size of point cloud object is calculated by clustering the coordinate, the existence time of object is calculated by matching the object in two frames, and in combination with object reflection intensity, if it is stable static object, alarm and object in reference monitoring surface coordinate are transmitted to visualization interface and displayed in monitoring area, to realize real-time visualization monitoring.

Description

Technical Field

[0001] This invention belongs to the field of power safety operation monitoring, three-dimensional visualization and proximity monitoring technology, specifically involving a real-time visualization monitoring method based on a virtual fence for proximity areas. Background Technology

[0002] In scenarios involving the inspection of high-voltage power lines and on-site operations at substations, the proximity of personnel and construction machinery to energized equipment can easily lead to safety accidents. The industry generally uses proximity monitoring and alarm methods to prevent risks, but traditional monitoring technologies have several obvious shortcomings:

[0003] 1. Relying solely on two-dimensional coordinates to delineate hazardous areas is not suitable for three-dimensional operation scenarios such as high-altitude and complex substations, and lacks the ability to control vertical spatial boundaries.

[0004] 2. Ordinary camera image recognition is severely affected by lighting, occlusion, and background interference, resulting in poor recognition accuracy and the inability to obtain the three-dimensional coordinates of intruding objects.

[0005] 3. The system is relatively independent and closed, and cannot be deeply integrated with the inspection platform; the web interface only supports two-dimensional data display and cannot intuitively configure the monitoring area in the three-dimensional scene; at the same time, the 3DGS high-precision three-dimensional reconstruction scene is not compatible with the conventional ray picking mechanism, and the virtual fence cannot be directly clicked to locate, making it difficult to implement three-dimensional configuration.

[0006] 4. Personal positioning alarm solution: It requires the installation of positioning hardware on the work object, which has high deployment and maintenance costs; the positioning accuracy is insufficient in indoor / obstructed environments, and it cannot identify the three-dimensional outline of the object and the degree of intrusion. The fence cannot be integrated with the real three-dimensional scene.

[0007] In summary, existing technologies are insufficient to achieve intuitive 3D scene fencing, 3D visualization of intrusion targets, full-link linkage of software and hardware, and reliable data management, thus failing to meet the high-precision and high-reliability near-electricity safety monitoring requirements of power sites. Summary of the Invention

[0008] To address the aforementioned technical problems, the present invention provides a method for real-time visual monitoring of near-electric zone virtual fences, comprising the following steps: Retrieve user-created monitoring scope instructions, including the monitoring scope and monitoring direction; Based on the current real-time location of the monitoring equipment, ray projection is performed in the 3DGS scene. The reference monitoring surface is obtained by combining the initial attitude parameters and the monitoring range. The outer warning area is obtained by extending unidirectionally along the monitoring direction. The monitoring area is delineated by combining the effective monitoring range of the lidar. Collect lidar monitoring data within the monitoring area, calculate the object point cloud data to obtain the object coordinates, and issue an alert if the object coordinates are within the outer warning area; otherwise, determine the object's coordinates within the reference monitoring surface. The coordinates are clustered to calculate the size of the point cloud objects. The objects within two frames are matched to calculate the existence time of the objects. Combined with the reflection intensity of the objects, if they are stable static objects, an alarm is triggered and the coordinates of the objects in the reference monitoring plane are transmitted to the visualization interface to be displayed in the monitoring area to achieve real-time visualization monitoring.

[0009] Preferably, based on the reference monitoring surface ID in the monitoring range instruction, the reference monitoring surfaces are added, deleted, updated, specified to be deleted, or all are deleted.

[0010] Preferably, after ray projection is performed on the 3DGS scene, the elevation is superimposed according to the preset value to generate the center point of the virtual fence, and the reference monitoring surface is obtained by combining the initial attitude parameters and the monitoring range.

[0011] Preferably, at the center point of the virtual fence, a rotation matrix is ​​generated by combining the initial attitude parameters and the monitoring range. A local coordinate system is constructed by performing a rotation transformation on the rotation matrix. Based on the local coordinate system, it is converted into the world coordinate system and then converted into latitude, longitude and height format to obtain the reference monitoring surface.

[0012] Preferably, when the reference monitoring surface and the extended early warning area are defined, the monitoring surface is rendered in a semi-transparent green to intuitively display the fence boundary and the direction of the lidar monitoring. When the parameters are modified, the scene model is refreshed in real time. Interactive adjustments support mouse dragging and directional key step movement of the fence. The drag offset is converted into an ENU coordinate system offset value, and the spatial position of the reference monitoring surface is updated in real time.

[0013] Preferably, the lidar monitoring data is collected based on a TCP / IP server according to the service interface parameters, the audible and visual alarm device is driven to issue the warning or alarm according to the serial port interface parameters, and the gimbal component is configured to transmit the coordinates of the object in the reference monitoring plane according to the gimbal information.

[0014] Preferably, voxel filtering is performed on the object coordinates, and the object's trajectory is determined based on the temporal changes in the object coordinates. If the object's trajectory indicates that the object is in a dynamic or static state and remains within the extended warning area, image analysis is performed in real time, and a similarity comparison is made with a preset warning object library. If the object is within the preset warning object library, a warning is issued and the comparison result is visualized.

[0015] Preferably, if the trajectory of the object indicates that the object dynamically passes through the extended warning area and then crosses the reference monitoring surface, or if the object is stationary within the reference monitoring surface, then the coordinates of the object within the reference monitoring surface are determined.

[0016] Preferably, the gimbal assembly includes a gimbal and a lidar. Based on the coordinates of the object in the reference monitoring plane, the gimbal assembly converts the coordinates of the object in the reference monitoring plane into the rotation angle of the gimbal, and the rotation of the gimbal drives the lidar to aim at the location of the intruding object.

[0017] Preferably, the coordinates of the object at the time of the warning are obtained and displayed in orange on the visualization interface, while the coordinates of the object within the reference monitoring surface are displayed in red on the visualization interface.

[0018] This invention provides a real-time visual monitoring method based on a virtual fence in a near-electrical zone. The method acquires user-created monitoring range instructions, including the monitoring range and direction. Based on the current real-time position of the monitoring device, ray projection is performed in the 3DGS scene. A reference monitoring surface is obtained by combining initial attitude parameters and the monitoring range. An extended warning area is obtained by unidirectionally extending the surface along the monitoring direction. The monitoring area is further defined by combining the effective monitoring range of the lidar. Lidar monitoring data is collected within the monitoring area, and object point cloud data is calculated to obtain object coordinates. If the object coordinates are within the extended warning area, an alarm is triggered; if they are within the reference monitoring surface, the object's coordinates within the reference monitoring surface are determined. The coordinates are clustered to calculate the object size in the point cloud. Objects within two frames are matched to calculate their existence time. Combined with the object's reflection intensity, if the object is a stable static object, an alarm is triggered. The object's coordinates within the reference monitoring surface are transmitted to the visualization interface via a pan-tilt system and displayed in the monitoring area to achieve real-time visual monitoring. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of a real-time visual monitoring method based on a virtual fence in the near-electric zone. Detailed Implementation

[0020] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0021] The terms and their definitions mentioned in this invention are as follows: The reference monitoring surface is a pre-calibrated virtual plane that serves as a rigid reference boundary for the system to determine intrusion behavior. There must be no obstructions such as rocks, equipment, or walls within the optical path from the radar to this virtual plane; obstructions will cause point cloud defects, leading to missed or false detections.

[0022] Extended warning zone: Based on the baseline monitoring surface, a three-dimensional buffer space extends unidirectionally in the monitoring direction to detect targets approaching the controlled boundary in advance, providing early warning prompts. As a preliminary transitional space for intrusion detection, the warning zone can capture the approach trend of targets in advance, extend the system response time, and realize a two-level hierarchical alarm mechanism of "early warning - intrusion alarm", adapting to the business needs of early intervention in security scenarios.

[0023] Monitoring direction: A spatial vector perpendicular to the plane of the electronic fence, representing the unidirectional discrimination dimension of the system's intrusion identification. The monitoring direction defines the warning area and the effective identification direction of the electronic fence. All advance warning and intrusion direction detection logic uses this vector as the judgment benchmark to distinguish whether the target is entering or leaving the controlled area, thus achieving unidirectional boundary control.

[0024] Clearance Processing: This is a pre-processing scene calibration operation performed before the system officially starts intrusion detection. Using the complete optical path from the LiDAR to the virtual plane of the electronic fence, and the fence plane covering the monitoring space as the processing scope, all non-permanent static obstructions and temporary obstacles within the scene are removed to construct an interference-free baseline background environment. After clearance processing, the point cloud acquired by the radar retains only the permanently fixed structures of the scene as the baseline background. This eliminates interference from temporary objects in the fixed point cloud, avoiding continuous false alarms and false intrusion alerts during the monitoring phase, and ensuring the reliability of subsequent target approach and boundary crossing intrusion detection results.

[0025] Monitoring Direction: This setting is used to indicate the warning area, alerting the user in advance if an object is approaching the monitored area in that direction. This feature addresses the common need in many monitoring tasks to "monitor entry but not exit" within a specific direction. To differentiate the threat posed by intruding objects to the monitored area, the monitoring area is divided into two levels: warning and alarm. The warning area is larger and extends in the monitoring direction, alerting the user to the approach of an object in that area.

[0026] Alarm Zone: A user-defined monitoring area used to notify the user that the area has been compromised.

[0027] This invention provides a real-time visualization monitoring method based on a virtual fence for near-electric zones, applicable to the web visualization configuration of virtual fences for near-electric zones in 3D Gaussian Splatting (3DGS) scenes, LiDAR slave device monitoring, and linked monitoring. The method includes the following steps: like Figure 1 As shown, obtain the monitoring range command created by the user. The specific initialization configuration and process are shown in steps 0-3.

[0028] Step 0: Initialize parameters, initialize subscribers / publishers / services, initialize TF (laser-baselink).

[0029] Initialization parameters include serial port interface parameters for driving the audible and visual alarm devices and service interface parameters for managing and controlling the monitoring equipment.

[0030] Step 1: Configure subscription data / service. Based on the monitoring range command created by the user, start the monitoring surface configuration service. If a reference monitoring surface is added, ray projection will be performed according to the current real-time location of the monitoring device and the 3DGS scene. The elevation will be superimposed according to the preset value to generate the center point of the virtual fence.

[0031] Subscribed data / services include baseline monitoring surface information, PTZ information, and TCP / IP server information.

[0032] The monitoring range command includes the reference monitoring surface ID, monitoring range, and monitoring direction.

[0033] The benchmark monitoring surface configuration service also includes: If a benchmark monitoring surface is added, the database is checked to see if the monitoring surface exists based on the benchmark monitoring surface ID. If it does not exist, the benchmark monitoring surface is created; if it exists, it is updated based on the benchmark monitoring surface ID. If a specified benchmark monitoring surface is deleted, the relevant data will be searched and deleted based on the benchmark monitoring surface ID. Deleting all benchmark monitoring surfaces will delete all benchmark monitoring surface data.

[0034] Step 2: Based on the center point of the virtual fence, generate a rotation matrix by combining the initial attitude parameters and the monitoring range in the monitoring range command. Perform a rotation transformation on the rotation matrix to construct a local coordinate system. Based on the local coordinate system, convert it to the world coordinate system, and then convert it to latitude, longitude and height format to obtain the reference monitoring surface, including the geographic coordinates, dimensions, orientation angle and pitch angle of the four corner points of the fence.

[0035] The initial attitude parameters are 10m×10m in size, 0° in heading angle, and 90° in pitch angle.

[0036] Step 3: Based on the monitoring direction in the monitoring range instruction, extend the reference monitoring surface unidirectionally along the monitoring direction to obtain the extended early warning area, encapsulate and bind the monitoring surface ID and monitoring direction, and visualize and render the reference monitoring surface and the extended early warning area.

[0037] During visualization rendering, the monitoring surface is rendered in a semi-transparent green, intuitively displaying the fence boundary and the orientation of the LiDAR monitoring. When parameters are modified, the scene model is refreshed in real time. Interactive adjustments support mouse dragging and arrow key step movement of the fence. The drag offset is converted into ENU coordinate system offset value, and the spatial position of the reference monitoring surface is updated in real time.

[0038] Step 4: Delineate the monitoring area on the reference monitoring surface and the extended early warning area according to the effective monitoring range of the lidar, such as... Figure 1 As shown.

[0039] like Figure 1 As shown, the coordinates of the corresponding objects detected by the lidar within the monitoring area are calculated. If the object is in the outer warning area, a warning is issued. If the object is on the reference monitoring surface, the coordinates of the object within the reference monitoring surface are determined, as shown in steps 5-7.

[0040] Step 5: Based on the TCP / IP server in the subscription data / service, collect LiDAR monitoring data within the monitoring area through the service interface parameters managed by the monitoring equipment, calculate the point cloud data of the object according to the principle of polar coordinates, and calculate the weighted average of the point coordinates of the point cloud data to obtain the object coordinates.

[0041] Step 6: Perform voxel filtering on the object coordinates. If the object coincides with a known voxel, retain it. Determine the object's trajectory based on the temporal changes of the retained object coordinates. If the object's trajectory indicates that the object is in a dynamic or static state within the extended warning area, perform real-time image analysis and compare it with a preset warning object library. If the object is within the preset warning object library, it is determined to be a valid target approaching the extended warning area without intruding into the reference monitoring surface. Drive the audible and visual alarm device based on the serial port interface parameters to issue an early warning and visualize the comparison results. Visualize the coordinates of the object within the extended warning area.

[0042] The pre-set warning object library is designed based on common objects in the work environment, storing images, geometric parameters, hazard levels, descriptions, and handling methods for common objects.

[0043] Step 7: If the trajectory of the object indicates that the object dynamically passes through the outer warning area and then crosses the reference monitoring surface, or if the object is stationary within the reference monitoring surface, determine the coordinates of the object within the reference monitoring surface.

[0044] like Figure 1 As shown, the point cloud object size is calculated by clustering the coordinates of the object in the reference monitoring surface, and stable static objects are selected and displayed on the visualization interface by combining the object's existence time and reflection intensity, as shown in steps 8-11.

[0045] Step 8: Cluster the coordinates of objects within the reference monitoring surface to calculate the size of the point cloud objects. By matching the identified objects in two frames one by one, calculate the existence time of the objects. Combine the object reflection intensity with the preset static object existence time range, spatial attribute range, and reflection intensity range to determine whether it is a stable static object. If so, it is determined to be a valid target intruding into the reference monitoring surface. Drive the sound and light alarm device to alarm based on the serial port interface parameters.

[0046] The steps for setting the duration of a preset static object are as follows: After the clearance process, the collected lidar monitoring data is analyzed for factors such as duration of existence, spatial properties of the object itself, and object reflection intensity. Based on the analysis results, preset time range, spatial attribute range, and reflection intensity range for static objects are determined.

[0047] Step 9: After an alarm is detected, based on the TCP / IP server in the subscribed data / service, the PTZ component, including the PTZ and LiDAR, is activated to obtain the location of the intruding object. Based on the object's coordinates in the reference monitoring plane, the PTZ component converts the object's coordinates in the reference monitoring plane into the PTZ rotation angle, and then the PTZ rotates to drive the LiDAR to aim at the location of the intruding object.

[0048] Step 10: Visualize the data from the PTZ component in the monitoring area, outlining the shape and location of the intruding object to achieve real-time visual monitoring.

[0049] Step 11: If the intrusion object is located in the outer warning area, determine whether the nearby mobile loading and unloading equipment can move the intrusion object outside the outer warning area by only contacting the part of the object in the outer warning area, based on the location and material of the intrusion object. If so, send a control command to the nearby mobile loading and unloading equipment.

[0050] The point cloud in the warning area is marked in orange, and the object that has invaded the baseline monitoring surface is marked in red. This visually displays the current alarm level of the invading object. Information such as the moving direction and intrusion time of the invading object is recorded in the system log, which effectively solves the problem that traditional monitoring methods cannot identify the three-dimensional outline of the object and the degree of intrusion.

[0051] The beneficial effects are as follows: 1. Overcome the industry challenge of not being able to pick coordinates in high-precision 3D scenes using 3DGS, and achieve intuitive click positioning of virtual fences in 3D scenes of power sites, greatly improving configuration accuracy and ease of operation.

[0052] 2. Enables full-parameter 3D visualization configuration and bidirectional coordinate calculation of the fence, supports drag-and-drop and button fine-tuning, adapts to complex 3D operation scenarios, and solves the limitations of traditional 2D fences and fixed fences.

[0053] 3. The lidar terminal completes static background filtering and multi-level noise filtering, effectively avoiding interference from static objects and flying points on site, and significantly improving the accuracy of intrusion identification; the hierarchical early warning + alarm mechanism meets the needs of power proximity safety management.

[0054] 4. It adopts asynchronous receipt, dual snapshot, and timeout protection mechanisms, configures data storage security, supports rollback of misoperation, and has multi-fence concurrent management capabilities to meet the multi-area monitoring needs of large substations.

[0055] 5. Enables real-time linkage of commands and data across the entire four-layer architecture, and provides a 3D point cloud visualization of the intrusion target, allowing maintenance personnel to intuitively determine the location and extent of the intrusion, thereby improving emergency response efficiency.

[0056] 6. The radar sensor is linked with the inspection equipment and visualized, allowing real-time viewing of the radar monitoring coverage area, making equipment maintenance and on-site debugging more convenient.

Claims

1. A method for real-time visual monitoring of near-electricity zones based on virtual fences, characterized in that, Includes the following steps: Retrieve user-created monitoring scope instructions, including the monitoring scope and monitoring direction; Based on the current real-time location of the monitoring equipment, ray projection is performed in the 3DGS scene. The reference monitoring surface is obtained by combining the initial attitude parameters and the monitoring range. The outer warning area is obtained by extending unidirectionally along the monitoring direction. The monitoring area is delineated by combining the effective monitoring range of the lidar. Collect lidar monitoring data within the monitoring area, calculate the object point cloud data to obtain the object coordinates, and issue an alert if the object coordinates are within the outer warning area; otherwise, determine the object's coordinates within the reference monitoring surface. The coordinates are clustered to calculate the size of the point cloud objects. The objects within two frames are matched to calculate the existence time of the objects. Combined with the reflection intensity of the objects, if they are stable static objects, an alarm is triggered and the coordinates of the objects in the reference monitoring plane are transmitted to the visualization interface to be displayed in the monitoring area to achieve real-time visualization monitoring.

2. The real-time visual monitoring method based on a virtual fence in a near-electric zone as described in claim 1, characterized in that, Based on the reference monitoring surface ID in the monitoring range instruction, the reference monitoring surfaces can be added, removed, updated, specified to be deleted, or all deleted.

3. The real-time visual monitoring method based on a virtual fence in a near-electric zone as described in claim 1, characterized in that, After ray projection is performed on the 3DGS scene, the elevation is superimposed according to the preset value to generate the center point of the virtual fence. The reference monitoring surface is obtained by combining the initial attitude parameters and the monitoring range.

4. The real-time visual monitoring method based on a virtual fence in a near-electric zone as described in claim 3, characterized in that, The virtual fence center point is generated, and a rotation matrix is ​​generated by combining the initial attitude parameters and the monitoring range. The rotation matrix is ​​then transformed to construct a local coordinate system. Based on the local coordinate system, it is converted to the world coordinate system and then converted to latitude, longitude and height format to obtain the reference monitoring surface.

5. The real-time visual monitoring method based on a virtual fence in a near-electric zone as described in claim 1, characterized in that, When the reference monitoring surface and the extended early warning area are defined, the monitoring surface is rendered in a semi-transparent green to intuitively display the fence boundary and the direction of the lidar monitoring. When the parameters are modified, the scene model is refreshed in real time. Interactive adjustments support mouse dragging and arrow key step movement of the fence. The drag offset is converted into an ENU coordinate system offset value, and the spatial position of the reference monitoring surface is updated in real time.

6. The real-time visual monitoring method based on a virtual fence in a near-electric zone as described in claim 1, characterized in that, Based on a TCP / IP server, the laser radar monitoring data is collected according to the service interface parameters. The sound and light alarm device is driven to issue the warning or alarm according to the serial port interface parameters. The pan-tilt component is configured to transmit the coordinates of the object in the reference monitoring plane according to the pan-tilt information.

7. The real-time visual monitoring method based on a virtual fence in a near-electric zone as described in claim 1, characterized in that, Voxel filtering is performed on the object coordinates. The object's trajectory is determined based on the temporal changes in the object coordinates. If the object's trajectory indicates that the object is in a dynamic or static state and remains within the extended warning area, image analysis is performed in real time, and a similarity comparison is made with a preset warning object library. If the object is in the preset warning object library, a warning is issued and the comparison result is visualized.

8. The real-time visual monitoring method based on a virtual fence in a near-electric zone as described in claim 7, characterized in that, If the trajectory of the object indicates that the object dynamically passes through the extended warning area and then crosses the reference monitoring surface, or if the object is stationary within the reference monitoring surface, then the coordinates of the object within the reference monitoring surface are determined.

9. The real-time visual monitoring method based on a virtual fence in a near-electric zone as described in claim 6, characterized in that, The gimbal assembly includes a gimbal and a lidar. Based on the coordinates of the object in the reference monitoring plane, the gimbal assembly converts the coordinates of the object in the reference monitoring plane into the rotation angle of the gimbal. The rotation of the gimbal then drives the lidar to aim at the location of the intruding object.

10. The real-time visual monitoring method based on a virtual fence in a near-electric zone as described in claim 1, characterized in that, The coordinates of the object at the time of the warning are obtained and displayed in orange on the visualization interface. The coordinates of the object within the reference monitoring plane are displayed in red on the visualization interface.

Images