A substation operation space virtual red line fence intelligent monitoring device and method

Abstract

The present invention discloses an intelligent monitoring device and method for a virtual red line fence in a substation operation space. The monitoring device is divided into four parts: data collection, data transmission, data processing, and mobile management platform. The monitoring method first uses the PnP (Perspective‑n‑Point) algorithm to realize the precise joint calibration of the camera and the laser radar, and constructs a three-dimensional space safety boundary around the key substation equipment-a virtual red line. Subsequently, relying on the collaborative processing technology of image and point cloud data, the rapid identification and distance judgment of intruding objects are realized. Once the object approaches or crosses the safety red line, the alarm mechanism is triggered. The present invention provides a set of efficient, flexible, and intelligent intelligent monitoring devices and systems for virtual red line fences in substation operation spaces, which can significantly reduce the safety risks caused by staff and construction equipment accidentally touching substation equipment, and provide new ideas and solutions for safe production and management in the power industry, with high engineering application value.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an intelligent monitoring device and method for a virtual red line fence of a substation operation space. Background Art

[0002] The power system, as the core pillar of the national energy system, is not only related to the power supply of people's daily life, but also an important guarantee for the normal production of industry and the security of national energy strategy. In this complex and sophisticated system, the substation plays a vital role. It is not only the focus of inspection and maintenance work, but also the key node of smart grid voltage regulation and power flow management, gathering and coordinating the massive data exchange inside and outside the power grid.

[0003] In recent years, safety accidents caused by accidental touch during the maintenance of substation equipment, especially casualties, have occurred from time to time, which highlights the urgency of strengthening safety protection. Traditionally, although infrared human body detection, field strength monitoring and other means can warn people approaching live equipment to a certain extent, they are limited by sensor accuracy, prone to false alarms or missed alarms, and lack the ability to intelligently identify target types. With the development of intelligent technology, substation anti-accidental touch solutions based on intelligent vision systems have gradually emerged. Through the fusion of cameras and sensors, accurate capture of equipment location and status can be achieved, significantly improving the efficiency of safety management and control. However, there are still problems such as monitoring blind spots that are difficult to solve by unlimited addition of cameras, and limited flexibility in system expansion and transformation. Therefore, how to improve the degree of automation and environmental adaptability while ensuring the accuracy of equipment detection, so as to make monitoring more efficient and convenient, is the key to solving the problem of applying virtual red line fence intelligent monitoring technology in actual scenarios. Summary of the invention

[0004] In view of this, the technical problem that the present invention actually aims to solve is: how to construct a set of intelligent monitoring devices and systems for virtual red line fences in substation operating space in a more compact, convenient and environmentally adaptable way, so as to make the safe production and management of the power industry more efficient, flexible and intelligent.

[0005] In view of this, the present invention provides a substation operation space virtual red line fence intelligent monitoring device and method, comprising the following steps:

[0006] Step (1) A set of intelligent monitoring device for virtual red line fence of substation operation space is designed, which consists of four parts: data collection, data transmission, data processing and mobile management platform;

[0007] Step (2) completes the precise joint calibration of the internal and external parameters of the camera and the lidar based on the PnP algorithm;

[0008] Step (3) using the calibrated camera to collect images, performing instance segmentation through the improved FCOS single-stage target detection network, and constructing the three-dimensional spatial safety boundary around the key substation equipment - the virtual red line;

[0009] Step (4) relies on the collaborative processing technology of image and point cloud data to achieve rapid identification and distance judgment of intruding objects. Once the object approaches or crosses the safety red line, the alarm mechanism is triggered.

[0010] Compared with the prior art, the present invention has the following beneficial effects:

[0011] (1) Through the independently developed intelligent algorithm for collaborative processing of images and point clouds, the processing and calculation efficiency and accuracy of scene three-dimensional data are greatly improved, meeting the needs of low latency, rapid response and alarm in power scenarios. It also makes up for the problems of LiDAR's inability to detect and identify targets, easy confusion between intruders and monitored target point cloud areas, as well as the low accuracy of binocular vision spatial positioning, fixed position of the camera and the target area, and small scope of application.

[0012] (2) The proposed segmentation method based on the FCOS single-stage object detection framework adds a bottom branch in the feature extraction stage of the FCOS object detection network to calculate the score of the feature map at the pixel granularity, adds a top-level attention branch after the Head module to calculate the global attention feature map within the output bounding box of each layer of the feature pyramid, and also fuses the feature maps of the bottom branch and the top branch. Compared with the two-stage instance segmentation method, its segmentation effect not only improves the segmentation accuracy but also reduces the calculation time.

[0013] (3) The developed intelligent monitoring device for virtual red line fence in substation operation space is small in size, ready for use as soon as it is powered on, and has strong environmental adaptability. It can be used for safety monitoring in different scenarios such as substation and transmission, different live equipment, and different angles, and has huge market application and promotion potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 The present invention is an overall flow chart of an intelligent monitoring device and method for a virtual red line fence in a substation operation space.

[0015] Figure 2 It is a schematic diagram of an intelligent monitoring device for a virtual red line fence of a substation operation space according to the present invention.

[0016] Figure 3 The present invention provides a flow chart for the joint calibration of internal and external parameters of an intelligent monitoring method for a virtual red line fence in a substation operation space.

[0017] Figure 4 The present invention provides a virtual red line construction flow chart of a method for intelligently monitoring a virtual red line fence in a substation operation space. DETAILED DESCRIPTION

[0018] The specific implementation methods of a virtual red line fence intelligent monitoring device and method for a substation operation space of the present invention are described in detail below in conjunction with the accompanying drawings.

[0019] like Figure 1 As shown, the present invention provides a device and method for intelligent monitoring of virtual red line fences in substation operation space, and the specific process is as follows:

[0020] Step (1) A set of intelligent monitoring device for virtual red line fence of substation operation space is designed, which consists of four parts: data collection, data transmission, data processing and mobile management platform, such as Figure 2 . Data collection includes the collection of visible light images and three-dimensional laser point clouds; data transmission includes image and point cloud data transmission and data processing information transmission; data processing includes the fusion of visible light images and point cloud data, the construction of virtual red lines, rapid detection of intruding objects and alarms for approaching objects, etc.; the mobile management platform collects data information transmitted back by safety control devices at different locations in the substation for unified storage, display and management. The specific steps are as follows:

[0021] (11) The data collection end uses cameras and lidar to work together. The camera, with its high resolution and high-speed transmission capabilities, captures clear images of power transmission and transformation equipment and its surrounding environment, providing rich information for subsequent image analysis and understanding. The lidar, with its long-distance measurement capabilities and robustness to low-reflectivity objects, accurately obtains three-dimensional point cloud data of the scene, providing an accurate spatial information foundation for building virtual safety red lines.

[0022] (12) Data transmission includes two transmission links, including image and point cloud data transmission and data processing information transmission. On the one hand, the original image and point cloud data are quickly transmitted to the processing box via wired means to reduce data delay and loss during transmission; on the other hand, the processing box, as an edge computing node, performs preliminary processing and analysis on the received data, and then transmits key information such as processing results, virtual safety red line status, and intrusion alarms to the mobile management platform in real time via the wireless network.

[0023] (13) Data processing includes camera and LiDAR calibration, single-stage target detection instance segmentation, and image-point cloud collaborative algorithms, which are used to build a virtual red line for substations and detect intruding objects. The system uses advanced algorithms to ensure that camera and LiDAR data are consistent, identify key equipment, and build a three-dimensional model. Image-point cloud collaboration improves perception capabilities, monitors foreign object intrusion in real time, and triggers alarms when approaching the virtual red line quickly, ensuring the safety of power transmission and transformation facilities.

[0024] (14) The mobile management platform realizes system networking through portable devices such as notebooks, uniformly stores, displays and manages data information, and displays data information from various monitoring points in real time. It also supports multi-device networking and remote control functions, allowing managers to grasp the operating status of the entire system anytime and anywhere, and flexibly adjust and optimize according to actual conditions.

[0025] Step (2) Based on the PnP algorithm, the camera and lidar internal and external parameters are precisely calibrated.

[0026] Please refer to Figure 3 First, the camera and lidar sensors are calibrated. The camera intrinsic calibration is completed by using the MATLAB single target calibration toolbox. The lidar intrinsic calibration is completed by aligning the timestamps of each point in the same frame point cloud and performing corresponding motion compensation according to the timestamps. Then, the camera and lidar joint calibration method based on the PnP algorithm is used to complete the external calibration. The specific steps are as follows:

[0027] (21) First, the intrinsic calibration of the camera and lidar sensors is performed. The intrinsic calibration of the monocular camera is completed using the MATLAB single-object calibration toolbox, and then the intrinsic calibration of the lidar is performed by aligning the timestamps of each point in the same frame point cloud and performing corresponding motion compensation based on the timestamps.

[0028] (22) The PnP algorithm is used to complete the precise joint calibration of the camera and the lidar. A method for automatically selecting 2D-3D matching point pairs is proposed. A specific calibration plate is used to convert the 3D point cloud into a 2D projection image, and then the edge of the 2D projection image is extracted. Finally, the calibration plate is segmented from the 3D point cloud. The specific steps for edge extraction of the 2D image captured by the camera are as follows:

[0029] (221) converting the visible light image into the HSV color space according to the color of the calibration plate, and then setting an appropriate color threshold to separate the calibration plate;

[0030] (222) Morphological operations are used to filter out redundant extracted discrete points and fill some small gaps in the calibration plate;

[0031] (223) Traverse all pixels of the binary image in the form of raster scanning, determine whether the current pixel belongs to a boundary point in turn, and start executing the boundary search algorithm from the boundary point, and finally obtain the outer contour and inner contour of the binary image. The outer contour and inner contour are given in the form of a point set, and the inner and outer contours of the calibration plate can be drawn in the RGB image based on the contour point set.

[0032] (224) In order to eliminate the error caused by inaccurate contours, the minimum circumscribed rectangle of the edge point set is further extracted. First, the convex hull of the point set is calculated, and then according to the geometric theorem that "one side of the minimum circumscribed rectangle of a polygon must be collinear with one of the sides of the polygon", each side of the convex hull is enumerated as a circumscribed rectangle, and the rectangle with the smallest area is selected as the required minimum circumscribed rectangle. The inner and outer contours of the calibration plate can be accurately segmented, and the two-dimensional image coordinates of the edge points can be obtained.

[0033] (23) For the three-dimensional point cloud image acquired by the lidar, the RANSAC (random sampling consensus) algorithm is first used to fit the calibration plate plane, and then the edge contour point set is extracted. The edge three-dimensional coordinates are obtained by an algorithm that calculates edge points based on existing prior knowledge. The specific steps are as follows:

[0034] (231) The RANSAC algorithm is used to separate the calibration plate plane from the point cloud. First, the number of iterations and the distance threshold of the RANSAC algorithm are determined. Within the number of iterations, three points (not collinear) are randomly selected to calculate the plane equation. The distances from all points to the plane are calculated. The number of "inliers" that are less than the set distance threshold is counted. The above steps are repeated until the number of iterations is exceeded or the number of "inliers" is greater than a certain threshold. Finally, the plane equation with the largest number of "inliers" is selected as the output.

[0035] The number of iterations is related to the ratio of "inside points" to "outside points" in the data. Assuming that the proportion of "inside points" in all discrete data is t:

[0036]

[0037] Then, in the process of selecting 3 points to calculate the plane equation, the probability of selecting at least one "outside point" is: 1-t n (n=3). In the case of k iterations, the probability that all three selected points are "interior points" is:

[0038] P=1-(1-t n ) k (2)

[0039] Where P can be interpreted as the probability of fitting a plane in k iterations, which can be set to 0.99, and k is derived as:

[0040]

[0041] (232) The least squares method is used to solve the point cloud normal vector of the calibration plate, and then the contour point sets of the four edges of the calibration plate are extracted based on the contour point set of the calibration plate point cloud. Finally, the RANSAC algorithm is used to fit the edge line equation to obtain the three-dimensional coordinates of the edge points.

[0042] The algorithm steps of the RANSAC algorithm for fitting a two-dimensional straight line are as follows:

[0043] ① Select two points from a set of edge points and calculate the equation of the line.

[0044] ② Calculate the distance from other points in the point set to the straight line. If the distance from a point to the straight line is less than the threshold, it is regarded as an “inside point”.

[0045] ③Repeat steps ① and ② to select the straight line equation with the largest number of internal points.

[0046] Assume that the slope of the straight line on the edge of the calibration plate is k, and the upper left corner P of the calibration plate lu The coordinates of the calibration plate are (x0, y0, z0). If the size of the calibration plate is known, then the coordinates of the upper right corner of the calibration plate are P ru (x2, y2, z2), the coordinates of the upper left corner of the hollow rectangle S lu (x4,y4,z4), coordinates of the top vertex C of the hollow circle u (x5,y5,z5) is calculated as:

[0047]

[0048]

[0049]

[0050] The x-coordinate values ​​of these three points are calculated based on the plane equation, and the calculation method of the remaining edge points is similar to the above three points.

[0051] (233) The extracted calibration plate image coordinates and point cloud coordinates are used as the input of the PnP algorithm to obtain the transformation matrix between the two coordinate systems, that is, the external parameters of the camera and lidar, and complete the joint calibration.

[0052] Step (3) using the calibrated camera to collect images, performing instance segmentation through the improved FCOS (Fully Convolutional One-Stage Object Detection, FCOS) single-stage object detection network, and constructing the three-dimensional space safety boundary around the key substation equipment - the virtual red line;

[0053] The improved FCOS single-stage target detection network is used to perform instance segmentation, find the corresponding three-dimensional information, and then through simple manual interaction, select any target area in the scene as the protection target of the virtual red line to be constructed. Combined with object recognition, the three-dimensional coordinates of dangerous power equipment in the image are calculated to complete the construction of the virtual safety red line of key equipment in the power transmission and transformation field. Figure 4 The specific steps are as follows:

[0054] (31) First, the original image X is input into the backbone network of the FCOS framework for full convolution feature extraction, and the feature maps C3, C4, and C5 are output. They are then fused through the feature pyramid network (FPN) to output five feature images of different sizes F3, F4, F5, F6, and F7, all of which are H×W in size.

[0055] (32) The output feature maps C3, C4, C5 of the backbone network or the output feature maps F3, F4, F5, F6, F7 of the feature pyramid are used as input, and the scores of the feature maps at the pixel granularity are calculated through the bottom-level branches, and the output result B is output, where N represents the batch size, K represents the number of B, H×W is the size of the input feature map of the bottom-level branch, and s is the step size of the convolutional layer in the bottom-level branch. The structure of the bottom-level branch is the same as the decoder of DeepLabV3+.

[0056] (33) The output feature maps F3, F4, F5, F6, and F7 of the feature pyramid are sent to the top-level attention branch for convolution to predict the attention map within the target detection bounding box. The output result A represents the size H l ×W l K·M·M channel feature map of .

[0057] Assume that the output size of the first layer of the FCOS framework Head network is H l ×W l , then the size of the attention map of the top-level attention branch at each position is N×(K·M·M)×H l ×W l , where M×M is the resolution of the attention map.

[0058] (34) Apply the post-processing operation of the FCOS framework to the feature map A of the top-level attention branch to select D target detection bounding boxes and its corresponding attention map A = {a d ∈R K×M×M |d=1...D}, eliminating redundant prediction boxes for the same object in the target detection method.

[0059] The post-processing operation uses the non-maximum suppression (NMS) method, that is, the probabilities of all predicted bounding boxes are sorted from high to low, and finally the predicted box with the highest probability is selected as the final output of the target detection network.

[0060] (35) The feature map B of the bottom branch, the attention map A of the top attention branch and its corresponding predicted bounding box P are input into the fusion module. The specific steps are as follows:

[0061] (351) The RoIPool operation in an improved object detection and instance segmentation algorithm (Mask Region-Based Convolutional Neural Network, Mask-RCNN) is used to crop the feature map B to the same size as the predicted bounding box P, and output a feature map of a fixed size of R × R. The process is expressed as follows:

[0062]

[0063] (352) Generally speaking, the size of the attention map A is M×M, which is smaller than R×R. Therefore, the attention map A is interpolated:

[0064]

[0065] (353) For the interpolated attention map a' d Apply softmax activation function to the K channels of s to regularize them and obtain a set of attention score maps s d :

[0066]

[0067] (354) Output r of RoIPool operation d and attention score map s d Multiply each pixel in K dimensions to get the mask image m d , instance segmentation can be completed:

[0068]

[0069] (36) The segmentation result is used as the protection target of the virtual red line to be constructed. According to its three-dimensional coordinates, the center of the object in space is calculated, and then the distance from the center of the object to each edge of the object is calculated respectively. A safety threshold is added to each distance to complete the construction of the virtual red line. For the sake of insurance, a virtual yellow line can be added as a warning line at a certain threshold distance outside the virtual red line.

[0070] Step (4) relies on the collaborative processing technology of image and point cloud data to achieve rapid identification and distance judgment of intruding objects. Once the object approaches or crosses the safety red line, the alarm mechanism is triggered.

[0071] After calculating the yellow and red safety lines, the system uses the sparse depth perception-based moving target overly close warning. When other objects such as workers, cranes or other construction equipment are detected in the image, their three-dimensional coordinates are calculated to determine whether the coordinates (x1, y1, z1) of the object have entered the yellow cuboid. If so, it means that the distance between the object and the power equipment is less than the safety threshold, which may cause danger, and a warning sound is issued; when the foreign object continues to approach the red cuboid, a rapid alarm sound will be issued immediately, indicating that it is very dangerous. The specific steps are as follows:

[0072] (41) An algorithm for feature point detection and description (Oriented FAST and RotatedBRIEF, ORB) is used to obtain feature point matching pairs in the inner area of ​​the moving target in the left image, and the sparse disparity values ​​are obtained using the matching point pairs. Then, the distance of the moving target is measured based on the principle of binocular stereo vision. The specific steps are as follows:

[0073] (411) Use FAST to locate a set of candidate features, and use the grayscale centroid method to add the corner point direction to make it have a description of rotation and scale, and calculate the centroid:

[0074]

[0075] Where (x, y) is the relative position of the feature point, r is the radius of the circular neighborhood, (x, y)∈[-r, r], I is the gray value of the coordinate (x, y) in the figure, m pq is the (p+q)-order moment of the feature point, and the centroid of the neighborhood of the FAST feature point can be obtained:

[0076]

[0077] (412) Assuming that the center of the feature point is O, the direction of the vector OC from its center to the grayscale centroid C is the direction of the feature point, that is, the direction of the FAST feature point, and the angle of the OC vector is the direction of the feature point:

[0078] θ=arctan(m 10 ,m 01 ) (13)

[0079] (413) Using BRIEF binary feature description, we randomly select n pairs of points in the neighborhood of a point P on the image, and compare the pixel values ​​of these pairs of points to obtain a string of binary results:

[0080]

[0081] Then the BRIEF descriptor can be represented by a binary vector and matched by the Hamming distance:

[0082]

[0083] (414) Since the binocular camera used in this device has been calibrated and corrected by Zhang's calibration method. The image planes of the two cameras after correction are completely coplanar with each other, have parallel optical axes, are at a known distance, and have equal focal lengths, and the feature points of the left and right cameras correspond to each other. In this case, the depth is inversely proportional to the parallax, and the depth Z can be calculated by similar triangles, where f is the focal length of the left and right cameras, P is the target point, d is the parallax, T is the baseline distance, and Z is the distance between the target point and the camera.

[0084] According to the principle of similar triangles, referring to the schematic diagram, we can get:

[0085]

[0086] You can get:

[0087]

[0088] Assume that one of the matching point pairs is: P{p l (x l ,y l ),p r (x r ,y r )}, then the disparity value of the matching point pair is: d = x l -x r , substituting the disparity value into formula (17) can obtain its three-dimensional space distance.

[0089] (42) By calculating the distance information of multiple feature points within the moving target and taking the average, the real spatial distance of the target can be obtained, and the distance is compared with the preset safety threshold. If the real spatial distance is less than the safety threshold, it is judged that the distance between the moving target (such as a worker, a crane or other construction equipment) and the power equipment is too close, and a safety warning sound is issued in time; if the moving target moves closer to the red rectangular area (representing an extremely dangerous area), the system will issue a rapid alarm sound to warn of potential danger.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit the present invention. A person skilled in the art should understand that the specific implementation of the present invention can be modified or replaced by equivalents, but these modifications or changes are within the scope of protection of the pending claims.

Claims

1. A method for intelligent monitoring of virtual redline fences in substation operation space, characterized in that: The following steps are involved: Step (1) A set of intelligent monitoring device for virtual red line fence of substation operation space is designed, which consists of four parts: data collection, data transmission, data processing and mobile management platform; Step (2) completes the precise joint calibration of the internal and external parameters of the camera and the lidar based on the PnP algorithm; Step (3) using the calibrated camera to collect images, performing instance segmentation through the improved FCOS single-stage target detection network, and constructing the three-dimensional spatial safety boundary around the key substation equipment - the virtual red line; Step (4) relies on the collaborative processing technology of image and point cloud data to achieve rapid identification and distance judgment of intruding objects. Once the object approaches or crosses the safety red line, the alarm mechanism is triggered.

2. According to claim 1, a method for intelligent monitoring of virtual redline fences in substation operation space is characterized in that: In step (1), a set of intelligent monitoring device for virtual red line fence of substation operation space is designed, which specifically includes the following steps: (11) The data collection end uses cameras and lidar to work together. The camera, with its high resolution and high-speed transmission capabilities, captures clear images of the power transmission and transformation equipment and its surrounding environment, providing rich information for subsequent image analysis and understanding. The lidar, with its long-distance measurement capabilities and robustness to low-reflectivity objects, accurately obtains three-dimensional point cloud data of the scene, providing an accurate spatial information foundation for constructing virtual safety red lines. (12) Data transmission includes two transmission links, including image and point cloud data transmission and data processing information transmission; on the one hand, the original image and point cloud data are quickly transmitted to the processing box via wired means to reduce data delay and loss during transmission; on the other hand, the processing box, as an edge computing node, performs preliminary processing and analysis on the received data, and then transmits the processing results, virtual security red line status, and intrusion alarm key information back to the mobile management platform in real time via the wireless network; (13) Data processing includes camera and lidar calibration, single-stage target detection instance segmentation, and image-point cloud collaborative algorithm, which is used to build a virtual red line of the substation and detect intruding objects. The system uses advanced algorithms to ensure the consistency of camera and lidar data, identify key equipment and build a three-dimensional model. Image-point cloud collaboration improves perception capabilities, monitors foreign object intrusion in real time, and triggers alarms when approaching the virtual red line quickly, ensuring the safety of power transmission and transformation facilities. (14) The mobile management platform realizes system networking through portable devices such as notebooks, uniformly stores, displays and manages data information, and displays data information from various monitoring points in real time; it also supports multi-device networking and remote control functions, allowing managers to grasp the operating status of the entire system anytime and anywhere, and flexibly adjust and optimize according to actual conditions.

3. According to claim 1, a method for intelligent monitoring of virtual redline fences in substation operation space is characterized in that: In step (2), the precise joint calibration of the internal and external parameters of the camera and the lidar is completed based on the PnP algorithm. The specific steps are as follows: (21) First, the intrinsic parameters of the camera and lidar sensors are calibrated. The intrinsic parameters of the monocular camera are calibrated using the MATLAB single-target calibration toolbox. Then, the intrinsic parameters of the lidar are calibrated by aligning the timestamps of each point in the same frame point cloud and performing corresponding motion compensation according to the timestamps. (22) The PnP algorithm is used to complete the precise joint calibration of the camera and the lidar. A method for automatically selecting 2D-3D matching point pairs is proposed. A specific calibration plate is used to convert the 3D point cloud into a 2D projection image, and then the edge of the 2D projection image is extracted. Finally, the calibration plate is segmented from the 3D point cloud. The specific steps of edge extraction for the 2D projection image captured by the camera are as follows: (221) converting the visible light image into the HSV color space according to the color of the calibration plate, and then setting an appropriate color threshold to separate the calibration plate; (222) Morphological operations are used to filter out redundant extracted discrete points and fill some small gaps in the calibration plate; (223) Traverse all pixels of the binary image in the form of raster scanning, determine whether the current pixel belongs to a boundary point in turn, and start executing the boundary search algorithm from the boundary point, and finally obtain the outer contour and inner contour of the binary image; wherein the outer contour and the inner contour are given in the form of a point set, and the inner and outer contours of the calibration plate can be drawn in the RGB image according to the contour point set; (224) In order to eliminate the error caused by inaccurate contours, the minimum circumscribed rectangle of the edge point set is further extracted. First, the convex hull of the point set is calculated, and then according to the geometric theorem "one side of the minimum circumscribed rectangle of a polygon must be collinear with one of the sides of the polygon", each side of the convex hull is enumerated as a circumscribed rectangle, and the rectangle with the smallest area is selected as the required minimum circumscribed rectangle. The inner and outer contours of the calibration plate can be accurately segmented, and the two-dimensional image coordinates of the edge points can be obtained; (23) For the three-dimensional point cloud image acquired by the lidar, the RANdom SAmple Consensus (RANSAC) algorithm is first used to fit the calibration plate plane, and then the edge contour point set is extracted. The three-dimensional coordinates of the edge are obtained by an algorithm that calculates the edge points based on existing prior knowledge. The specific steps are as follows: (231) The RANSAC algorithm is used to separate the calibration plate plane from the point cloud; first determine the number of iterations and distance threshold of the RANSAC algorithm, randomly select three points (not collinear) within the number of iterations to calculate the plane equation, calculate the distance from all points to the plane, and count the number of "inliers" that are less than the set distance threshold. Repeat the above steps until the number of iterations is exceeded or the number of "inliers" is greater than a certain threshold, and finally select the plane equation with the largest number of "inliers" as the output; The number of iterations is related to the ratio of "inside points" to "outside points" in the data. Assuming that the proportion of "inside points" in all discrete data is t: Where n inliers represents "inside points", that is, points whose distance to the hypothesized plane is less than the set distance threshold, n outliers represents "inside points", that is, points whose distance to the hypothetical plane is greater than the set distance threshold; then in the process of selecting 3 points to calculate the plane equation, the probability of selecting at least one "outside point" is: 1-t n (n=3); In the case of k iterations, the probability that all three selected points are "interior points" is: P=1-(1-t n ) k (2) Where P is interpreted as the probability of fitting a plane in k iterations, which is set to 0.99, and k is derived as: (232) The least squares method is used to solve the point cloud normal vector of the calibration plate, and then the contour point sets of the four edges of the calibration plate are extracted according to the contour point set of the calibration plate point cloud. Finally, the RANSAC algorithm is used to fit the edge line equation to obtain the three-dimensional coordinates of the edge points. The algorithm steps of the RANSAC algorithm for fitting a two-dimensional straight line are as follows: ① Select two points from a certain edge point set and calculate the equation of the line; ② Calculate the distance from other points in the point set to the straight line. If the distance from a point to the straight line is less than the threshold, it is regarded as an "inner point"; ③ Repeat steps ① and ② to select the straight line equation with the largest number of interior points; Assume that the slope of the straight line on the edge of the calibration plate is k, and the upper left corner P of the calibration plate lu The coordinates are (x0, y0, z0); if the size of the calibration plate is known, then the coordinates of the upper right corner of the calibration plate are P ru (x2, y2, z2), the coordinates of the upper left corner of the hollow rectangle S lu (x4,y4,z4), coordinates of the top vertex C of the hollow circle u (x5,y5,z5) is calculated as: The x-coordinate values ​​of these three points are calculated based on the plane equation, and the calculation method of the remaining edge points is similar to the above three points; (233) The extracted calibration plate image coordinates and point cloud coordinates are used as the input of the PnP algorithm to obtain the transformation matrix between the two coordinate systems, that is, the external parameters of the camera and lidar, and complete the joint calibration.

4. According to claim 1, a method for intelligent monitoring of virtual redline fences in substation operation space is characterized in that: In step (3), a three-dimensional space safety boundary around the key substation equipment, namely a virtual red line, is constructed. The specific steps are as follows: (31) First, the original image X is input into the backbone network of the FCOS framework for full convolution feature extraction, and the feature maps C3, C4, and C5 are output. Then, they are fused through the feature pyramid network (FPN) to output five feature images of different sizes F3, F4, F5, F6, and F7, all of which are of size H×W. (32) The output feature maps C3, C4, C5 of the backbone network or the output feature maps F3, F4, F5, F6, F7 of the feature pyramid are used as input, and the scores of the feature maps at the pixel granularity are calculated through the bottom-level branches, and the output result B is output, where N represents the batch size, K represents the number of B, H×W is the size of the input feature map of the bottom-level branch, and s is the step size of the convolutional layer in the bottom-level branch; the structure of the bottom-level branch is the same as the decoder of DeepLabV3+; (33) The output feature maps F3, F4, F5, F6, and F7 of the feature pyramid are sent to the top-level attention branch for convolution to predict the attention map within the target detection bounding box. The output result A represents the size H l ×W l K·M·M channel feature map; Assume that the output size of the first layer of the FCOS framework Head network is H l ×W l , then the size of the attention map of the top-level attention branch at each position is N×(K·M·M)×H l ×W l , where M×M is the resolution of the attention map; (34) Apply the post-processing operation of the FCOS framework to the feature map A of the top-level attention branch to select D target detection bounding boxes and its corresponding attention map A = {a d ∈R K×M×M |d=1...D}, eliminating redundant prediction boxes for the same object in the target detection method; The post-processing operation uses the non-maximum suppression (NMS) method, which sorts the probabilities of all predicted bounding boxes from high to low, and finally selects the predicted box with the highest probability as the final output of the target detection network; (35) The feature map B of the bottom branch, the attention map A of the top attention branch and its corresponding predicted bounding box P are input into the fusion module. The specific steps are as follows: (351) The RoIPool operation in an improved object detection and instance segmentation algorithm (Mask Region-Based Convolutional Neural Network, Mask-RCNN) is used to crop the feature map B to the same size as the predicted bounding box P, and output a feature map of a fixed size of R × R. The process is expressed as follows: (352) Generally speaking, the size of the attention map A is M×M, which is smaller than R×R. Therefore, the attention map A is interpolated, where interpolate refers to a certain interpolation method: (353) For the interpolated attention map a' d Apply softmax activation function to the K channels of s to regularize them and obtain a set of attention score maps s d : (354) Output r of RoIPool operation d and attention score map s d Multiply each pixel in K dimensions to get the mask image m d , instance segmentation can be completed: (36) The segmentation result is used as the protection target of the virtual red line to be constructed. According to its three-dimensional coordinates, the center of the object in space is calculated, and then the distance from the center of the object to each edge of the object is calculated respectively. A safety threshold is added to each distance to complete the construction of the virtual red line. For the sake of safety, a virtual yellow line is added at a certain threshold distance outside the virtual red line as a warning line.

5. According to claim 1, a method for intelligent monitoring of virtual redline fences in substation operation space is characterized in that: In step (4), rapid identification and distance judgment of the intruding object are achieved. The specific steps are as follows: (41) An algorithm for feature point detection and description (Oriented FAST and Rotated BRIEF, ORB) is used to obtain feature point matching pairs in the inner area of ​​the moving target in the left image, and the sparse disparity values ​​are obtained using the matching point pairs. Then, the distance of the moving target is measured based on the principle of binocular stereo vision. The specific steps are as follows: (411) Use FAST to locate a set of candidate features, and use the grayscale centroid method to add the corner point direction to make it have a description of rotation and scale, and calculate the centroid: Where (x, y) is the relative position of the feature point, r is the radius of the circular neighborhood, (x, y)∈[-r, r], I is the gray value of the coordinate (x, y) in the figure, m pq is the (p+q)-order moment of the feature point, where p and q represent the order of the moment, i.e., the number of times the gray value is multiplied by its x and y coordinates; the centroid of the neighborhood of the FAST feature point can be obtained: (412) Assuming that the center of the feature point is O, the direction of the vector OC from its center to the grayscale centroid C is the direction of the feature point, that is, the direction of the FAST feature point, and the angle of the OC vector is the direction of the feature point: θ=arctan(m 10 ,m 01 ) (13) (413) Using BRIEF binary feature description, we randomly select n pairs of points in the neighborhood of a point P on the image, and compare the pixel values ​​of these pairs of points to obtain a string of binary results: The BRIEF descriptor is represented by a binary vector and matched by the Hamming distance: (414) Since the binocular camera used in this device has been calibrated and corrected by Zhang's calibration method; the image planes of the two cameras after correction are completely coplanar with each other, have parallel optical axes, are at a known distance, and have equal focal lengths, and the feature points of the left and right cameras correspond to each other; in this case, the depth is inversely proportional to the parallax, and the depth Z can be calculated by similar triangles, where f is the focal length of the left and right cameras, P is the target point, d is the parallax, T is the baseline distance, and Z is the distance between the target point and the camera; According to the principle of similar triangles, referring to the schematic diagram, we can get: You can get: Assume that one of the matching point pairs is: P{p l (x l ,y l ),p r (x r ,y r )}, then the disparity value of the matching point pair is: d = x l -x r , Substituting the disparity value into formula (17) can obtain its three-dimensional space distance; (42) By calculating the distance information of multiple feature points within the moving target and taking the average, the real spatial distance of the target can be obtained, and the distance is compared with the preset safety threshold. If the real spatial distance is less than the safety threshold, it is judged that the distance between the moving target and the power equipment is too close, and a safety warning sound is issued in time; if the moving target gets closer to the red rectangular area, the system will issue a rapid alarm sound to warn of potential danger.

Images