Holding pole attitude abnormity real-time monitoring and early warning method based on vision

Abstract

The invention provides a vision-based real-time monitoring and early warning method for abnormal postures of a holding pole. Comprising the following steps: (1) acquiring a field real-time video of a suspended derrick tower; (2) extracting picture data from the video for preprocessing; (3) detecting and positioning the derrick in the image by using an image segmentation algorithm to obtain derrick image information; (4) applying an attitude estimation algorithm to the detected derrick to estimate attitude parameters of the derrick; and (5) setting a reasonable attitude range, comparing and analyzing the attitude parameters of the holding pole, and judging whether an abnormal attitude exists or not. By means of the method, posture abnormity of the suspension derrick can be rapidly and accurately monitored and early warned, potential risks are recognized in advance, and reliability and stability of the derrick are ensured. And meanwhile, whether foreign matters are attached or not can be detected while the posture abnormity of the holding pole is detected, and the problems of insulation short circuit and the like are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of attitude estimation, and in particular to a vision-based real-time monitoring and early warning method for abnormal pole holding attitude. Background Art

[0002] The pole is a structure used to support antennas, wires or other equipment. It is commonly found in utility poles and is an important part of the power transmission and distribution system. In the construction of high-voltage towers, the posture of the suspended pole is very important for the construction and subsequent maintenance of the entire tower assembly method. Especially in complex mountainous terrain and changeable weather conditions, accurately judging the posture of the suspended pole and issuing danger warnings are crucial to ensuring the safety and reliability of high-voltage transmission line tower assembly.

[0003] With the development of society and the improvement of people's living standards, electricity plays an important role in daily life. As a key component of the power system, ultra-high voltage transmission lines have extremely high requirements for equipment and have a significant impact on people's electricity use and safety. Therefore, it is necessary to monitor the status of the pole in real time so that when it tilts, it can be discovered in time and respond quickly to reduce property losses and maintain the safety and stability of high-voltage power transmission. At present, the main solution to the tilt of the pole is to rely on manual direct inspection or manual identification through video monitoring.

[0004] The Chinese patent with publication number CN117278881A discloses a "system and method for monitoring the construction site of a suspended pole tower in a transmission line", which is characterized in that a plurality of tension sensors, including an alarm and a host, are placed in the internal suspended pole tower; the measuring component is used to obtain tension data and a first data transceiver unit for signal transmission; the alarm is used to send a warning signal; the host system includes a second data transceiver unit that communicates wirelessly with the first data transceiver unit, and detects the pole status in real time through data processing and triggers a warning message. This technical solution breaks through the traditional manual inspection or video monitoring method, and uses sensors for detection, which has better sensitivity and accuracy. However, the entire process relies on the measuring component. When the sensor fails due to extremely bad weather, it may be impossible to detect, which poses a certain safety risk. At the same time, the combination of a large number of sensors makes the cost relatively high. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a vision-based real-time monitoring and early warning method for abnormal pole posture, which solves the problems in the prior art of being difficult to detect under harsh working conditions and causing safety hazards, as well as the problem of high cost.

[0006] According to an embodiment of the present invention, a vision-based real-time monitoring and early warning method for abnormal pole holding posture includes the following steps:

[0007] S1. Obtain the on-site real-time video of the suspended pole tower assembly and extract the image data;

[0008] S2, preprocessing the image data;

[0009] S3. Detect and locate the pole in the image using an image segmentation algorithm to obtain pole image information;

[0010] S4, applying a posture estimation algorithm to the detected pole to estimate the posture parameters of the pole;

[0011] S5. Compare and analyze the posture parameters of the pole to determine whether there is an abnormal posture. If there is no abnormality, the system will continue to work normally. If there is an abnormality, the system will trigger the early warning and alarm mechanism, issue a voice warning, and send it remotely to the staff.

[0012] Furthermore, the step S1 is specifically as follows:

[0013] S1.1. Install multiple cameras according to the current position of the pole to ensure that the cameras can fully cover the pole;

[0014] S1.2. Continuously shoot the video of the pole with a camera, extract frames from the video at a certain frame rate, and obtain an image set of the current pole.

[0015] Furthermore, the optimal threshold of the image is selected through the Bayesian criterion and wavelet transform, and the wavelet coefficient value of the noise image is compared with the optimal threshold. Pixels below the threshold are normal pixels, and pixels above the threshold are noise pixels. They are expanded and processed to denoise the image and enhance image information.

[0016] Furthermore, the processed image is input into the trained detection model U-Net network module. Based on U-Net, the channel and spatial attention mechanism and residual module are introduced. The attention module plays a key role in pole feature extraction. The weight vector suitable for task requirements is learned through spatial and channel compression and channel excitation, and applied to the original feature map to enhance feature expression ability. The residual module is introduced to further enhance network performance, obtain context information and learn complex feature representation through jump connections, solve gradient problems and prevent overfitting.

[0017] Furthermore, the relationship between the three-dimensional space point and the actual imaging plane is calculated through the pinhole imaging principle.

[0018] Furthermore, the pixel coordinates are first converted into image coordinates corresponding to the two-dimensional space to achieve the mapping of each pixel point to the actual three-dimensional space point; each pixel in the image is traversed, and the pixels with the same adjacent pixel values ​​are marked as the same connected area to form a connected area of ​​the pole and its related parts. The minimum connected component area threshold is set to filter out connected areas with an area greater than the threshold. A mask image is generated based on the screening results. The image scanning algorithm based on deep search starts scanning from the upper left corner pixel of the image, marks the pixel as visited and adds it to the vector of the contour point; for the currently scanned pixel point, check the adjacent pixels around it. If the adjacent pixel is a point on the contour and has not been visited, it is marked as visited and added to the vector of the contour point; according to the intersection point of the pole and the traction rope, the coordinates of the center point of the pole are obtained.

[0019] Furthermore, by traversing all the points on the contour, the leftmost, topmost, rightmost, and bottommost points are found to obtain the minimum circumscribed rectangle of the contour; the midpoints of the sides of the rectangle are defined as the four endpoints of the pole contour, and the three-dimensional bounding box of the pole is obtained in combination with the camera imaging principle; the attitude estimation method of the center point and the three-dimensional bounding box is combined to define the final estimated attitude of the pole.

[0020] Furthermore, the step S5 is specifically as follows:

[0021] S5.1. According to the special orientation and application requirements of the pole, the staff shall set a reasonable posture range to determine whether it is tilted or deviates from the vertical, whether it is damaged or deformed, whether the surface is worn and rusted, or has other attachments;

[0022] S5.2. Compare and analyze the detected pole holding posture parameters with the set posture range to determine whether there is any abnormality. If there is any abnormality, a warning message will be issued and the push message will be sent to the staff and the person in charge.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] 1. The present invention obtains the posture information of the suspended pole tower assembly through real-time video. Even in the case of complex terrain and many weather change factors, the posture of the pole can be detected and located through image processing and analysis algorithms. The posture information of the suspended pole is obtained through real-time video, so that the state change of the pole can be monitored in time, abnormal posture can be discovered in advance, potential safety problems can be avoided, and whether there is an abnormal posture can be determined. Therefore, it has better environmental adaptability, will not cause inaccurate monitoring due to harsh working conditions, better guarantees real-time monitoring of the pole, improves the monitoring efficiency and accuracy of abnormal pole posture, and ensures the safe construction and later maintenance of the high-voltage transmission line tower assembly.

[0025] 2. In the present invention, after an abnormal posture is discovered, the system will trigger an early warning and alarm mechanism, issue a voice warning message in a timely manner, and push it to relevant staff, so as to timely discover the abnormality and take appropriate measures to reduce the complexity and cost of system maintenance; in addition, it can also determine whether there are foreign objects attached to the pole tower through image segmentation algorithm, avoid short-circuit failures of high-voltage lines, and reduce the time and cost of maintenance and troubleshooting. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of the real-time monitoring and early warning method in an embodiment of the present invention.

[0027] Figure 2 This is a network structure diagram for adding attention residual to U-Net.

[0028] Figure 3 It is a schematic diagram of the pixel coordinate system.

[0029] Figure 4 It is a schematic diagram of two-dimensional space coordinates. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0031] like Figure 1 As shown, the embodiment of the present invention proposes a vision-based real-time monitoring and early warning method and device for abnormal pole holding posture, comprising the following steps:

[0032] (1) Obtaining real-time video of the suspended pole tower assembly and extracting image data, specifically including the following sub-steps:

[0033] (1.1) Install multiple cameras based on the current position of the pole to ensure that the cameras can fully cover the area of ​​the pole.

[0034] (1.2) Use a camera to shoot a video of the suspended pole tower assembly, extract frames from the video at a certain frame rate, and obtain an image set of the current pole.

[0035] (2) Preprocessing the image data, specifically including the following sub-steps:

[0036] (2.1) De-noise the image and enhance the image information. Establish a threshold function. There are two types of threshold functions. The acquisition process is shown in the following formula:

[0037]

[0038] The wavelet coefficients of the image are described in t form, the threshold is described in G, the soft threshold function of the image is calibrated as α(t), the hard threshold function is expressed as β(t), and the sign function is expressed as sgn(t).

[0039] (2.2) According to the distribution characteristics of the image wavelet coefficients, the Bayesian criterion is used to select the optimal threshold of the image. The formula is as follows:

[0040]

[0041] The optimal threshold mark G(ε signal ) form, the noise variance estimation of the image signal is described The image signal variance is expressed as ε signal .

[0042] Assume the pixel position of the restored image is (i, j), and the gray value is described as g i，j , in order to obtain the noise image representation of the restored image, the process is as follows:

[0043] {Z i,j =g i,j +δ i,j ,i,j=1,2,…,m,m∈M}

[0044] The restored image noise is labeled M(·) with normal distribution, and the image noise is labeled δ i，j , the noise image is represented by Z i，j mark.

[0045] (2.3) Perform wavelet exchange on the above determined image and the noise image to obtain the wavelet coefficient value of the noise image. The result is as follows:

[0046]

[0047] The wavelet coefficients of the noise image are set as Original image wavelet coefficient value representation Noise wavelet coefficient description Finally, the optimal threshold is selected by the following formula to complete the image noise removal. The optimal threshold selection formula is as follows:

[0048]

[0049] (2.4) The selected threshold is compared with the image wavelet coefficient. Pixels below the threshold are normal pixels, while pixels above the threshold are noise pixels. The pixels are expanded and processed to complete the image denoising.

[0050] (3) Using an image segmentation algorithm to detect and locate the pole in the image and obtain pole image information, the specific steps include:

[0051] (3.1) The processed image is input into the trained detection model U-Net network module, and the channel and spatial attention mechanism and residual module are introduced based on U-Net.

[0052] (3.2) The attention residual structure is as follows Figure 2 As shown in the figure. The attention module plays a key role in pole feature extraction. It learns weight vectors suitable for task requirements through space and channel compression and channel excitation, and applies them to the original feature map to enhance feature expression capabilities. The residual module is introduced to further enhance network performance. It obtains context information and learns complex feature representations through jump connections, solves gradient problems and prevents overfitting.

[0053] (3.3) The network model includes 4 downsampling modules and 4 upsampling modules for pole image segmentation. The downsampling module consists of an attention residual structure and a maximum pooling layer, which is used to extract pole features and compress the feature map size. The upsampling module consists of a transposed convolution layer, which is used to upsample and retain context information. After being processed by the attention residual module and the 1×1 convolution layer, the final pole image segmentation result is obtained.

[0054] (4) Applying a posture estimation algorithm to the detected pole to estimate the posture parameters of the pole, specifically including the following sub-steps:

[0055] (4.1) The relationship between a three-dimensional space point and the actual imaging plane is calculated using the pinhole imaging principle.

[0056] like Figure 3 , Figure 4 As shown, taking the uov plane in the pixel coordinate system as the reference, the pixel coordinates need to be converted into the image coordinates corresponding to the two-dimensional space. Assume that the size of each pixel is dx, dy, and the pixel corresponding to the far point o in the image coordinate system is (u0, v0). By giving any specific pixel coordinates (u, v), the image coordinates can be calculated. The calculation formula is as follows:

[0057]

[0058] The spatial depth information obtained further can realize the conversion of two-dimensional image to three-dimensional spatial coordinates. According to the spatial calculation model of the camera, the formula is as follows:

[0059]

[0060] Where f is the focal length of the camera, z p is the depth information of the actual space point P in the camera coordinate system, thereby calculating the spatial coordinates of the actual space point in the camera coordinate system.

[0061] Define the focal length of the camera on the x and y axes as f x , f y , the relationship between pixel coordinates and spatial three-dimensional coordinates is as follows:

[0062]

[0063] Realize the mapping of each pixel point to the actual three-dimensional space point.

[0064] (4.2) A, B, C, D are the top left of the pole, the top right of the pole, the bottom left of the pole, the bottom right of the pole, and the center of the pole, respectively. Q is the center point of the pole. It has pixel coordinates (u a , v a ), (u b , v b ), (u c , v c ), (u d , v d ) and (u q , v q ), according to the above formula, the mapping between pixel points and actual three-dimensional space points can obtain the three-dimensional coordinates of A, B, C, D and Q as follows (X a , Y a , Z a ), (X b , Y b , Z b ), (X c , Y c , Z c ), (X d , Y d , Z d ) and (X q , Y q , Z q ), this method converts dz c Approximately dx c , the calculation formula is as follows:

[0065]

[0066] The detected target position and attitude information can be determined by the 3D bounding box. Let P1, P2, ..., P8 be the 8 vertices of the 3D bounding box of the pole, and the coordinates are (X i , Y i , Z i ), i = 1, 2, ... 8. Its coordinates can be obtained from Pi to Q relative geometry, such as the three-dimensional coordinates of P1 are:

[0067]

[0068] (4.3) In order to obtain the spatial posture of the pole, this method traverses each pixel in the image and marks the pixels with the same adjacent pixel values ​​as the same connected region, thus forming a connected region of the pole and its related parts. A minimum connected component area threshold is set to filter out connected regions with an area greater than the threshold. A mask image is generated based on the screening results.

[0069] (4.4) The image scanning algorithm based on depth search starts scanning from the upper left corner pixel of the image, marks the pixel as visited and adds it to the vector of contour points. For the currently scanned pixel, check the neighboring pixels around it. If the neighboring pixel is a point on the contour and has not been visited, it is marked as visited and added to the vector of contour points. According to the intersection point of the pole and the traction rope, the coordinates of the pole center point and the pole attitude estimation point are obtained. The coordinate calculation formula is as follows:

[0070]

[0071] Among them, f(x, y) is the pixel value of the binary image at point (x, y), x, y are the pixel coordinates, x c ,y c is the center coordinate of the pole. In order to obtain the coordinates of the pole intersection, all points on the traction rope contour are traversed, and the distance to the pole center point is calculated. The contour point with the minimum distance is the pole posture estimation point. The center point distance formula is as follows:

[0072]

[0073] where x i ,y i is the coordinate of the point on the contour, and the vector formed by the pole center point and the pole attitude estimation point is:

[0074]

[0075] The estimated posture q1 of the pole can be obtained. By traversing all points on the contour and comparing the x and y coordinates of each point, the leftmost, topmost, rightmost, and bottommost points are found to obtain the minimum circumscribed rectangle of the contour. The midpoints of each side of the rectangle are defined as the four endpoints of the pole contour, and combined with the camera imaging principle, the three-dimensional bounding box of the pole is obtained.

[0076] (4.5) In order to obtain a more accurate final posture of the pole, the three-dimensional bounding box of the pole is fused with the above-mentioned estimated posture. The coordinates of the center points of the six faces of the three-dimensional bounding box can be obtained through the position information of the three-dimensional bounding box of the pole. The distance between the pole posture estimation point and the center points of each face of the three-dimensional bounding box is calculated, and the vector formed by the point with the largest distance and the pole posture estimation point is defined as the pole posture q2.

[0077] (4.6) Combining the center point and 3D bounding box posture estimation methods, the two vectors q1 and q2 are fused to define the final estimated posture of the pole.

[0078] (5) Compare and analyze the posture parameters of the pole to determine whether there is an abnormal posture. If there is no abnormality, the system will continue to work normally. If there is an abnormality, the system will trigger the early warning and alarm mechanism, issue a voice warning, and send it to the staff remotely. The specific steps include the following:

[0079] (5.1) According to the special orientation and application requirements of the pole, the staff will set a reasonable posture range based on experience to determine whether it is tilted or deviates from the vertical, whether it is damaged or deformed, whether the surface is worn and rusted, or has other attachments, etc.

[0080] (5.2) Compare and analyze the detected pole holding posture parameters with the set posture range to determine whether there is any abnormality. If there is an abnormality, a warning message will be issued and the push information will be sent to the staff and the person in charge.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A vision-based real-time monitoring and early warning method for abnormal pole posture, characterized in that: The steps include: S1. Obtain the on-site real-time video of the suspended pole tower assembly and extract the image data; S2, preprocessing the image data; S3. Detect and locate the pole in the image using an image segmentation algorithm to obtain pole image information; S4, applying a posture estimation algorithm to the detected pole to estimate the posture parameters of the pole; S5. Compare and analyze the posture parameters of the pole to determine whether there is an abnormal posture. If there is no abnormality, the system will continue to work normally. If there is an abnormality, the system will trigger the early warning and alarm mechanism, issue a voice warning, and send it remotely to the staff.

2. The real-time monitoring and early warning method according to claim 1, characterized in that: The step S1 is specifically as follows: S1.

1. Install multiple cameras according to the current position of the pole to ensure that the cameras can fully cover the pole; S1.

2. Continuously shoot the video of the pole with a camera, extract frames from the video at a certain frame rate, and obtain an image set of the current pole.

3. The real-time monitoring and early warning method according to claim 1, characterized in that: The step S2 is specifically as follows: the optimal threshold of the image is selected through the Bayesian criterion and wavelet transform, and the wavelet coefficient value of the noise image is compared with the optimal threshold. Pixels below the threshold are normal pixels, and pixels above the threshold are noise pixels. The pixels are expanded and processed to denoise the image and enhance the image information.

4. The real-time monitoring and early warning method according to claim 1, characterized in that: The step S3 is specifically as follows: input the processed image into the trained detection model U-Net network module, introduce the channel and space attention mechanism and the residual module based on U-Net, the attention module plays a key role in the pole feature extraction, learns the weight vector suitable for the task requirements through space and channel compression and channel excitation, and applies it to the original feature map to enhance the feature expression ability. The residual module is introduced to further enhance the network performance, obtain context information and learn complex feature representation through jump connections, solve the gradient problem and prevent overfitting.

5. The real-time monitoring and early warning method according to claim 1, characterized in that: The step S4 specifically includes: calculating the relationship between the three-dimensional space point and the actual imaging plane through the pinhole imaging principle.

6. The real-time monitoring and early warning method according to claim 5, characterized in that: First, the pixel coordinates are converted into image coordinates corresponding to the two-dimensional space to realize the mapping of each pixel point to the actual three-dimensional space point; traverse each pixel in the image, mark the pixels with the same adjacent pixel values ​​as the same connected area, and form the connected area of ​​the pole and its related parts. Set the minimum connected component area threshold to filter out the connected areas with an area greater than the threshold. Generate a mask image based on the screening results. The image scanning algorithm based on deep search starts scanning from the upper left corner pixel of the image, marks the pixel as visited and adds it to the vector of the contour point; for the currently scanned pixel point, check the adjacent pixels around it. If the adjacent pixel is a point on the contour and has not been visited, it is marked as visited and added to the vector of the contour point; according to the intersection point of the pole and the traction rope, the coordinates of the center point of the pole are obtained.

7. The real-time monitoring and early warning method according to claim 6, characterized in that: By traversing all points on the contour, we find the leftmost, topmost, rightmost, and bottommost points to get the minimum circumscribed rectangle of the contour. We define the midpoints of each side of the rectangle as the four endpoints of the pole contour and obtain the three-dimensional bounding box of the pole based on the camera imaging principle. We define the final estimated posture of the pole based on the posture estimation method of the center point and the three-dimensional bounding box.

8. The real-time monitoring and early warning method according to claim 1, characterized in that: The step S5 is specifically as follows: S5.

1. According to the special orientation and application requirements of the pole, the staff shall set a reasonable posture range to determine whether it is tilted or deviates from the vertical, whether it is damaged or deformed, whether the surface is worn and rusted, or has other attachments; S5.

2. Compare and analyze the detected pole holding posture parameters with the set posture range to determine whether there is any abnormality. If there is any abnormality, a warning message will be issued and the push message will be sent to the staff and the person in charge.

Images