Method for monitoring the safe distance between large construction machinery and working area in substation

Abstract

The present invention discloses a method for monitoring the safe distance between large-scale construction machinery and the work area in a substation. The method comprises the following steps: collecting images of power construction work, establishing a target detection model, and using the target detection model to perform target detection on the images; extracting edge pixels of the detected targets and converting the pixel coordinates of the edge pixels into world coordinates; dividing the work area and the power outage area, and determining the danger zone; and performing safe distance monitoring based on the relationship between the large-scale construction machinery and the danger zone, in combination with the distance between the edge of the gantry support in the work area and the outer contour edge of the large-scale construction machinery. This method prevents people or large-scale construction machinery from accidentally entering the energized interval and causing the risk of electric shock, thereby effectively improving the safety and power supply reliability of the substation during non-power outage construction.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and relates to a method for monitoring the safe distance between large-scale construction machinery in a substation and a working area. Background Art

[0002] As user equipment demands increase power and substation power supply reliability requirements continue to rise, live working and non-stop operations have become crucial means of ensuring safe and reliable substation operation. Consequently, power companies have developed detailed safety regulations to ensure the smooth implementation of live working. Currently, the safety of live working relies primarily on the proactive work of construction workers and the oversight of dedicated supervisors. In practice, complex work environments, fatigue from prolonged work, and distracted workers can lead to violations of safety regulations, neglect of safe distances, and accidental entry into live compartments, resulting in accidents. Research on safe distance detection, both domestically and internationally, has primarily focused on electric field monitoring and image ranging monitoring. Electric field monitoring calculates the electric field strength of the vector electric field around power equipment and establishes a field strength-distance relationship to estimate the safe distance between large construction machinery and live objects. However, in actual construction, factors such as high-voltage field distortion caused by construction machinery and the poor environmental adaptability of monitoring equipment can result in low measurement accuracy and a short service life. This can lead to false alarms, missed alarms, and frequent maintenance for electric field monitoring devices. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for monitoring the safe distance between large-scale construction machinery in a substation and a working area, thereby solving the problem of low measurement accuracy in the prior art.

[0004] The technical solution adopted by the present invention is a method for monitoring the safe distance between large-scale construction machinery in a substation and a working area, comprising the following steps:

[0005] Collect pictures of power construction, establish a target detection model, and use the target detection model to perform target detection on the pictures of power construction; extract the edge pixels of the detection target and convert the pixel coordinates of the edge pixels into world coordinates; divide the working area and the power outage area, and determine the dangerous area; based on the relationship between large construction machinery and the dangerous area, combined with the distance between the edge of the gantry support in the working area and the outer contour edge of the large construction machinery, conduct safety distance monitoring.

[0006] The present invention is also characterized in that:

[0007] The following steps are involved:

[0008] Step 1: Collect power construction pictures and establish a power construction dataset;

[0009] Step 2: Build a target detection model and use it to perform target detection on power construction pictures;

[0010] Step 3: Extract the edge pixels of the detection target through the Canny edge extraction algorithm;

[0011] Step 4: Input the power construction dataset into the PSMNet network to generate a depth map for each power construction image and obtain the disparity value d of each pixel; solve the reprojection matrix Q based on the internal and external parameters of the ZED camera, and convert the pixel coordinates of the edge pixels into world coordinates according to the disparity value d of each pixel and the reprojection matrix Q;

[0012] Step 5: Use the gantry support as the basis for division to divide the working area and the power outage area;

[0013] Step 6: Based on the horizontal axis pixel coordinates of the gantry pillars, the gantry pillars are sorted from left to right, and the gantry pillars on the left and right sides of the working area are used as the basis for dividing the dangerous area, and red shadows are generated in the dangerous area;

[0014] Step 7: Calculate the Euclidean distance between the edge of the gantry support in the work area and the outer edge of the large construction machinery, and use the minimum value as the safety distance;

[0015] Step 8. If the pixel coordinates of the large construction machinery are outside the red shadow, the safety distance between the two is monitored in real time, and the large construction machinery with a gradually decreasing safety distance is marked; if the large construction machinery is in the red shadow, the Euclidean distance between the edge of the gantry support and the edge of the outer contour of the large construction machinery is greater than the safety distance, and the depth difference between the two is also greater than the safety distance, a prompt is issued; if the depth difference between the two is less than the safety distance, an alarm is issued.

[0016] Step 2 includes the following steps:

[0017] Step 2.1, add the attention mechanism SENet network to the feature channel of the yolov5 target detection model to obtain the target detection model;

[0018] Step 2.2: Label the gantry support and large construction machinery in the power construction images; input the labeled power construction images into the target detection model for training to obtain the optimal target detection model;

[0019] Step 2.3: Use the optimal target detection model to identify targets in the power construction dataset and obtain detection targets, which include gantry supports and large construction machinery.

[0020] Step 3 includes the following steps:

[0021] Step 3.1: Convert the input image into a binary image containing the object's contour information based on the Canny edge detection algorithm. Horizontally and vertically traverse the binary image to obtain the edge coordinates of the large construction machinery, the vertical coordinates of the gantry crossarm, and the horizontal coordinates of the pillar edge.

[0022] Step 3.2: Take the center coordinates of the pillar at the crossarm as the starting point, set the center point coordinates to (h, w), use (h, w) as the center coordinates of the h+1 row, and use this as the starting point to traverse horizontally to obtain the left and right edge coordinates of the pillar h+1 row, and use the left and right edge coordinates of the h+1 row to calculate the center point coordinates of the pillar h+1 row, thereby constructing a loop structure; when the center value of the h+k row is greater than the starting left and right edge horizontal coordinates, exit the loop and output the left and right edge coordinate matrix of the gantry pillar.

[0023] In step 4, the reprojection matrix Q is solved based on the internal and external parameters of the ZED camera:

[0024]

[0025] Where T is the baseline of the camera, f is the focal length of the camera, (c x ,c y ) is the origin of the camera coordinate system;

[0026] According to the disparity value d of each pixel, the reprojection matrix Q converts the pixel coordinates of the edge pixels into world coordinates:

[0027]

[0028] Where (x, y) is the image pixel coordinate, and X, Y, and Z are the three-dimensional world coordinates.

[0029] Step 5 includes the following steps:

[0030] Step 5.1: Use the ZED binocular camera and infrared imager to capture the same area and obtain binocular images and infrared thermal images.

[0031] Step 5.2: Match and fuse the same contents in the binocular image and the infrared thermal image based on the photothermal fusion algorithm to obtain a fused image, and establish a connection between the temperature value and the pixel value of each pixel coordinate in the fused image;

[0032] Step 5.3: Set a pixel value threshold to distinguish the working area from the power-off area. Based on the pixel value threshold, traverse the fused image to keep the pixel value of the working area unchanged and set the pixel value of the power-off area to zero, and obtain the outer contour coordinates of the working area.

[0033] The Euclidean distance between the edge of the gantry support in the work area and the outer contour edge of the large construction machinery in step 7 is calculated as follows:

[0034]

[0035] In the formula, (X s , Y s , Z s ) is the three-dimensional coordinate of the outer contour of large construction machinery, (X z , Y z , Z z ) is the three-dimensional coordinate of the outer contour of the gantry support.

[0036] In step 7, the outer contour of the large construction machinery is divided into left and right parts based on the center point, and the pixels on the edge of the large construction machinery contour closest to the edge of the gantry support are selected. The Euclidean distance between the aforementioned large construction machinery contour edge pixels and the edge of the gantry support in the working area is calculated.

[0037] The calculation formula for the depth difference in step 8 is:

[0038] D depth =|Z s -Z Z | (5).

[0039] The beneficial effects of the present invention are: the method for monitoring the safe distance between large-scale construction machinery and the working area of ​​a substation of the present invention is based on the advantages of rich semantic information of pictures or videos, good ranging range and long service life, and proposes a method for monitoring the safe distance between large-scale construction machinery and the working area of ​​a substation based on binocular stereo matching and edge segmentation fusion. It is used for monitoring the safe distance between large-scale construction machinery and the live area during live operations in the substation, preventing people or large-scale construction machinery from accidentally entering the live interval and causing the risk of electric shock, thereby effectively improving the safety and power supply reliability of the substation during non-stop construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of a method for monitoring the safe distance between large-scale construction machinery and a working area in a substation according to the present invention;

[0041] Figure 2 This is a flow chart of edge coordinate extraction in the method for monitoring the safe distance between large-scale construction machinery and the working area of ​​a substation according to the present invention;

[0042] Figure 3 This is a flowchart for dividing the working area and the power outage area in the safety distance monitoring method between large-scale construction machinery and the working area of ​​a substation. DETAILED DESCRIPTION

[0043] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Example 1

[0045] The method for monitoring the safe distance between large-scale construction machinery and the working area of ​​a substation includes the following steps:

[0046] Collect pictures of power construction, establish a target detection model, and use the target detection model to perform target detection on the pictures of power construction; extract the edge pixels of the detection target and convert the pixel coordinates of the edge pixels into world coordinates; divide the working area and the power outage area, and determine the dangerous area; based on the relationship between large construction machinery and the dangerous area, combined with the distance between the edge of the gantry support in the working area and the outer contour edge of the large construction machinery, conduct safety distance monitoring.

[0047] Example 2

[0048] Method for monitoring the safe distance between large construction machinery and working areas in substations, such as Figure 1 As shown, the following steps are included:

[0049] Step 1: Collect power construction pictures and establish a power construction dataset;

[0050] Step 2: Build a target detection model and use it to perform target detection on power construction pictures;

[0051] Step 3: Extract the edge pixels of the detection target through the Canny edge extraction algorithm;

[0052] Step 4: Input the power construction dataset into the PSMNet network to generate a depth map for each power construction image and obtain the disparity value d of each pixel; solve the reprojection matrix Q based on the internal and external parameters of the ZED camera, and convert the pixel coordinates of the edge pixels into world coordinates according to the disparity value d of each pixel and the reprojection matrix Q;

[0053] Step 5: Use the gantry support as the basis for division to divide the working area and the power outage area;

[0054] Step 6: Based on the horizontal axis pixel coordinates of the gantry pillars, the gantry pillars are sorted from left to right, and the gantry pillars on the left and right sides of the working area are used as the basis for dividing the dangerous area, and red shadows are generated in the dangerous area;

[0055] Step 7: Calculate the Euclidean distance between the edge of the gantry support in the work area and the outer edge of the large construction machinery, and use the minimum value as the safety distance;

[0056] Step 8. If the pixel coordinates of the large construction machinery are outside the red shadow, the safety distance between the two is monitored in real time, and the large construction machinery with a gradually decreasing safety distance is marked; if the large construction machinery is in the red shadow, the Euclidean distance between the edge of the gantry support and the edge of the outer contour of the large construction machinery is greater than the safety distance, and the depth difference between the two is also greater than the safety distance, a prompt is issued; if the depth difference between the two is less than the safety distance, an alarm is issued.

[0057] Example 3

[0058] The method for monitoring the safe distance between large-scale construction machinery and the working area of ​​a substation includes the following steps:

[0059] Step 1: Use the ZED binocular camera to collect power construction images and establish a power construction dataset. Considering the complexity of power construction scenes, mosaic data enhancement is used on the power construction images. The power construction images are spliced ​​through random scaling, arbitrary cropping, and random distribution to enhance the model's ability to recognize detection targets in complex environments.

[0060] Step 2: Build a target detection model and use it to perform target detection on power construction pictures;

[0061] Step 2.1. Add the attention mechanism SENet network to the feature channel of the yolov5 target detection model to further improve the model's detection accuracy for designated targets, while suppressing the detection of non-designated targets. Select the CIoU_Loss+DIoU_nms method to solve the problems of multi-target box screening and target occlusion overlap, and obtain the target detection model.

[0062] Step 2.2: Label the gantry support columns and large construction machinery in the power construction images in the power construction dataset; input the labeled images into the target detection model for training to obtain the optimal target detection model;

[0063] Step 2.3: Use the optimal target detection model to identify targets in the power construction dataset and obtain detection targets, including gantry supports and large construction machinery.

[0064] Step 3: Use the Canny edge extraction algorithm to extract the edge pixels of the detection target; Figure 2 As shown,

[0065] Step 3.1: Convert the input image into a binary image containing the object's contour information based on the Canny edge detection algorithm. Horizontally and vertically traverse the binary image to obtain the edge coordinates of the large construction machinery, the vertical coordinates of the gantry crossarm, and the horizontal coordinates of the pillar edge.

[0066] Step 3.2: Take the center coordinates of the pillar at the crossarm as the starting point, set the center point coordinates to (h, w), use (h, w) as the center coordinates of the h+1 row, and use this as the starting point to traverse horizontally to obtain the left and right edge coordinates of the pillar h+1 row, and use the left and right edge coordinates of the h+1 row to calculate the center point coordinates of the pillar h+1 row, thereby constructing a loop structure; when the center value of the h+k row is greater than the starting left and right edge horizontal coordinates, exit the loop and output the left and right edge coordinate matrix of the gantry pillar.

[0067] Step 4: Input the power construction dataset into the PSMNet network to generate a depth map for each power construction image, obtaining the disparity value d for each pixel. Compared to traditional stereo matching methods, the PSMNet network does not require any post-processing. Instead, it integrates global environmental information into image features using a pyramid pooling module and uses a stacked hourglass 3D CNN to generate a depth map. The intrinsic and extrinsic parameter matrices of the ZED camera are obtained using the Zhang calibration method. The reprojection matrix Q is solved based on the internal and external parameters of the ZED camera. The pixel coordinates of the edge pixels are converted to world coordinates based on the disparity value d and the reprojection matrix Q for each pixel.

[0068] Specifically, an end-to-end stereo matching network PSMNet is constructed, and the SceneFlow dataset is used to train the PSMNet network to obtain the optimal PSMNet network. The power construction dataset is input into the optimal PSMNet network to generate a depth map of each power construction image, and then the disparity value d of each pixel is obtained; the reprojection matrix Q is solved based on the internal and external parameters of the ZED camera, see formula (1), and the pixel coordinate system of the edge pixel is converted to the world coordinate system according to the disparity value d of each pixel and the reprojection matrix Q, see formula (2);

[0069]

[0070] Where T is the baseline of the camera, f is the focal length of the camera, (c x ,c y ) is the origin of the camera coordinate system;

[0071]

[0072] Where x and y are the image pixel coordinates, and X, Y, and Z are the three-dimensional world coordinates.

[0073] Step 5: Based on the principle that equipment in the working area generates heat due to loss during operation and equipment in the power outage area does not generate heat, the gantry support is used as the basis for division, and the working area and the power outage area are divided by using the fusion method of visible light imaging and infrared imaging; Figure 3 As shown,

[0074] Step 5.1: Use the ZED binocular camera and infrared imager to capture the same area and obtain binocular images and infrared thermal images.

[0075] Step 5.2: Match and fuse the same contents in the binocular image and the infrared thermal image based on the photothermal fusion algorithm to obtain a fused image, and establish a connection between the temperature value and the pixel value of each pixel coordinate in the fused image;

[0076] Step 5.3: Set a pixel value threshold to distinguish the working area from the power-off area. Based on the pixel value threshold, traverse the fused image to keep the pixel value of the working area unchanged and set the pixel value of the power-off area to zero, and obtain the outer contour coordinates of the working area.

[0077] Step 6: Based on the horizontal axis pixel coordinates of the gantry pillars in the image, the gantry pillars are sorted from left to right, and the gantry pillars on the left and right sides of the working area are used as the basis for dividing the dangerous area, and red shadows are generated in the dangerous area;

[0078] Step 7: Based on the center point of the large construction machine, its outer contour is divided into left and right parts. The pixel closest to the gantry support edge in the large construction machine outline is selected. The Euclidean distance between the gantry support edge in the working area and the outer contour edge of the large construction machine is calculated according to formula (4), and the minimum value is used as the safety distance.

[0079]

[0080] Where, (X s , Y s , Z s ) is the three-dimensional coordinate of the outer contour of large construction machinery, (X z , Y z , Z z ) is the three-dimensional coordinate of the outer contour of the gantry support.

[0081] Step 8: If the pixel coordinates of the large construction machinery are outside the red shadow, the safety distance between the two is monitored in real time, and the large construction machinery whose safety distance gradually decreases is marked; if the large construction machinery is in the red shadow, the Euclidean distance between the edge of the gantry support and the outer edge of the large construction machinery is greater than the safety distance, and the depth difference between the two is also greater than the safety distance, a prompt is issued; if the depth difference between the two is less than the safety distance, an alarm is issued; the depth difference is calculated as follows:

[0082] D depth =|Z s -Z Z | (5).

[0083] Through the above method, the present invention proposes a method for monitoring the safe distance between large-scale construction machinery and the working area of ​​a substation based on binocular stereo matching and edge segmentation fusion, based on the advantages of rich semantic information in pictures or videos, good ranging range and long service life. The method is used to monitor the safe distance between large-scale construction machinery and the working area of ​​the substation during live operations in the substation, to prevent people or large-scale construction machinery from accidentally entering the live interval and causing the risk of electric shock, thereby effectively improving the safety and power supply reliability of the substation during non-stop construction.

Claims

1. A method for monitoring the safe distance between large-scale construction machinery and the working area of ​​a substation, characterized in that: The following steps are involved: Collect images of power construction, establish a target detection model, and use the target detection model to perform target detection on the images. Extract edge pixels of the detected targets and convert the pixel coordinates of these edge pixels into world coordinates. Divide the work area and power outage area, and identify the danger zone. Based on the relationship between large construction machinery and the danger zone, and the distance between the edge of the gantry support in the work area and the outer contour edge of the large construction machinery, monitor the safety distance. The following steps are involved: Step 1: Collect power construction pictures and establish a power construction dataset; Step 2: Build a target detection model and use it to perform target detection on power construction pictures; Step 3: Extract edge pixels of the detection target using the Canny edge extraction algorithm; Step 4: Input the power construction dataset into the PSMNet network to generate a depth map of each power construction image and obtain the disparity value of each pixel. d ; Solve the reprojection matrix based on the internal and external parameters of the ZED camera Q , according to the disparity value of each pixel d , reprojection matrix Q Converting the pixel coordinate system of the edge pixel into world coordinates; Step 5: Use the gantry support as the basis for division to divide the working area and the power outage area; Step 6: Based on the horizontal axis pixel coordinates of the gantry pillars, the gantry pillars are sorted from left to right, and the gantry pillars on the left and right sides of the working area are used as the basis for dividing the dangerous area, and red shadows are generated in the dangerous area; Step 7: Calculate the Euclidean distance between the edge of the gantry support in the work area and the outer edge of the large construction machinery, and use the minimum value as the safety distance; Step 8: If the pixel coordinates of the large construction machinery are outside the red shadow, the safety distance between the two is monitored in real time, and large construction machinery with a gradually decreasing safety distance is marked. If the large construction machinery is within the red shadow, the Euclidean distance between the edge of the gantry support and the outer edge of the large construction machinery is greater than the safety distance, and the depth difference between the two is also greater than the safety distance, a prompt is issued. If the depth difference between the two is less than the safety distance, an alarm is issued. Step 3 includes the following steps: Step 3.1: Convert the input image into a binary image containing the target object's contour information based on the Canny edge detection algorithm, and obtain the edge coordinates of the large construction machinery, the vertical coordinates of the gantry crossarm, and the horizontal coordinates of the pillar edge by horizontally and vertically traversing the binary image; Step 3.2: Starting from the center coordinates of the crossarm pillar, set the center coordinates to (h, w). Using (h, w) as the center coordinates of row h+1, traverse horizontally from this starting point to obtain the left and right edge coordinates of the pillar in row h+1. Use the left and right edge coordinates of row h+1 to calculate the center coordinates of the pillar in row h+1, thus constructing a loop structure. When the center value of row h+k is greater than the starting left and right edge horizontal coordinates, exit the loop and output the left and right edge coordinate matrix of the gantry pillar. Step 5 includes the following steps: Step 5.1: Use the ZED binocular camera and infrared imager to capture the same area and obtain binocular images and infrared thermal images. Step 5.2: Match and fuse the same contents in the binocular image and the infrared thermal image based on the photothermal fusion algorithm to obtain a fused image, and establish a relationship between the temperature value and the pixel value of each pixel coordinate in the fused image; Step 5.3: Set a pixel value threshold to distinguish the working area from the power-off area, traverse the fused image based on the pixel value threshold to keep the pixel value of the working area unchanged, set the pixel value of the power-off area to zero, and obtain the outer contour coordinates of the working area.

2. The method for monitoring the safe distance between large-scale construction machinery and the working area of ​​a substation according to claim 1 is characterized in that: Step 2 includes the following steps: Step 2.1, add the attention mechanism SENet network to the feature channel of the yolov5 target detection model to obtain the target detection model; Step 2.2: Label the gantry support and large construction machinery in the power construction pictures; input the labeled power construction pictures into the target detection model for training to obtain the optimal target detection model; Step 2.3: Use the optimal target detection model to identify targets in the power construction dataset and obtain detection targets, which include gantry supports and large construction machinery.

3. The method for monitoring the safe distance between large-scale construction machinery and the working area of ​​a substation according to claim 1 is characterized in that: In step 4, the reprojection matrix is ​​solved based on the internal and external parameters of the ZED camera Q : （1）； Where, T is the camera baseline, f is the camera focal length, ( c x ,c y ) is the origin of the camera coordinate system; According to the disparity value of each pixel d , reprojection matrix Q Convert the pixel coordinates of the edge pixels to world coordinates: （2）； Where, ( x , y ) is the image pixel coordinate , X, Y, Z 3D world coordinates.

4. The method for monitoring the safe distance between large-scale construction machinery and the working area of ​​a substation according to claim 1 is characterized in that: The Euclidean distance between the edge of the gantry support in the work area and the edge of the outer contour of the large construction machinery in step 7 is calculated as follows: （4）； Where, ( X s , Y s , Z s ) is the three-dimensional coordinate of the outer contour of large construction machinery, ( X z , Y z , Z z ) is the three-dimensional coordinate of the outer contour of the gantry support.

5. The method for monitoring the safe distance between large-scale construction machinery and the working area of ​​a substation according to claim 4 is characterized in that: In step 7, the outer contour of the large construction machinery is divided into left and right parts based on the center point, and the pixels on the edge of the large construction machinery contour closest to the edge of the gantry support are selected. The Euclidean distance between the aforementioned large construction machinery contour edge pixels and the edge of the gantry support in the working area is calculated.

6. The method for monitoring the safe distance between large-scale construction machinery and the working area of ​​a substation according to claim 4, characterized in that: The calculation formula for the depth difference in step 8 is: （5）。

Images