A method and system for sensing foreign object targets in the vicinity of a power transmission line

Abstract

The present application relates to the technical field of image processing, in particular to a kind of foreign object target perception method and system of power transmission line periphery, comprising: obtaining the power transmission line and crane in each frame monitoring image, obtain the depth distortion degree of each pixel point on the power transmission line plane, according to the change of crane boom pixel point in adjacent frame monitoring image, obtain all frame monitoring image of crane in each time hoisting movement, and then obtain the correction distance of each crane boom pixel point and each pixel point on the power transmission line in each frame monitoring image, obtain the plane of crane in each time hoisting movement and the width height distortion influence factor of crane in each time hoisting movement, obtain the protection measure of power transmission line.The present application aims to solve the problem that the distance between power transmission line and crane boom in image and actual distance have difference due to the influence of near big and far small on object in image when image is photographed by using camera.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method and system for sensing foreign objects around power transmission lines. Background Technology

[0002] The power transmission process has long been plagued by various safety threats. Damage to transmission lines is mainly caused by climate change, the growth of trees around the lines, and damage during construction. Since the possibility of damage to transmission lines during construction can be reduced through human intervention, it is necessary to conduct real-time detection and sensing of foreign objects around the transmission lines within the construction area.

[0003] Currently, the detection of foreign objects around power transmission lines within a construction area relies on cameras installed on high-voltage towers to capture and monitor the lines. The potential for damage is assessed by analyzing the distance between the crane boom and the power lines in the images. However, due to pixel distortion (objects appearing larger when closer and smaller when farther away) during image capture, the actual distance between the crane boom and the power lines is not accurately reflected in the images, thus affecting the accuracy of foreign object detection around power transmission lines. Summary of the Invention

[0004] The present invention provides a method and system for detecting foreign objects around power transmission lines, which adopts the following technical solution:

[0005] This invention proposes a method for detecting foreign objects around power transmission lines, which includes the following steps:

[0006] Take fixed distances from different power transmission lines and continuous frame monitoring images of power transmission lines and cranes;

[0007] Based on the distribution of pixels on different transmission lines in each frame of the monitoring image and the fixed distance between each transmission line and other transmission lines, the degree of depth distortion of the plane where each pixel on the transmission line is located is obtained.

[0008] Based on the changes in the crane boom pixels in adjacent frame monitoring images, the probability of the crane performing a lifting motion during each operation is obtained, thus obtaining continuous frame monitoring images of the crane performing a lifting motion.

[0009] Based on the distance between each crane boom pixel and each pixel on the power transmission line in the monitoring image, and the degree of depth distortion of the plane where each pixel on the power transmission line is located, the corrected distance between each crane boom pixel and each pixel on the power transmission line in each frame of the monitoring image is obtained. Based on the sequence of corrected distances between different crane boom pixels and the same pixel on the power transmission line in consecutive frames of the monitoring image during the crane's lifting motion, the final distance between the crane boom pixels and the power transmission line in each frame of the monitoring image during each lifting motion is obtained, and this is used to detect foreign objects around the power transmission line.

[0010] Furthermore, the specific method for obtaining the depth distortion degree of the plane containing each pixel on the power transmission line based on the distribution of pixels on different power transmission lines within each frame of the monitoring image and the fixed distance between each power transmission line and other power transmission lines includes:

[0011] The first On the power transmission line and the first The first on the power transmission line The pixel closest to the nth pixel is used as the nth pixel. The first power transmission line The pixel at the th point The corresponding pixels of each transmission line;

[0012] The specific formula for calculating the depth distortion of the plane containing each pixel on the transmission line is as follows:

[0013]

[0014] In the formula, Indicates the first The first on the power transmission line The degree of depth distortion in the plane containing each pixel. Indicates the first The first on the power transmission line The pixel and its position in the first... The distance between each transmission line and the corresponding pixel. Indicates the first The power transmission line and the first A fixed distance between power transmission lines.

[0015] Furthermore, the specific method for determining the probability of a crane performing a lifting motion during each operation based on changes in the crane boom pixels within adjacent monitoring frames includes:

[0016] All monitoring images are arranged in ascending order of their corresponding time to obtain a monitoring image sequence. The frame difference method is used to compare the crane boom in each monitoring image in the monitoring image sequence with the crane boom in the next monitoring image. If the crane boom pixels in any two adjacent monitoring images are completely one-to-one, the two adjacent monitoring images are recorded as crane stopped moving images.

[0017] Within the monitoring image sequence, the image of the crane stopping motion is used as the segmentation point. All frame monitoring images are divided into multiple image segments. All frame monitoring images in each image segment are used as all frame monitoring images during one operation of the crane.

[0018] Based on the differences in the distribution of crane boom pixels in consecutive frame monitoring images during each operation, the probability of the crane performing a lifting motion during each operation can be determined.

[0019] Furthermore, the specific method for determining the probability of a crane performing a lifting motion during each operation based on the distribution differences of crane boom pixels in consecutive frame monitoring images during each operation includes:

[0020] Using the pixel at the bottom left corner of the monitored image as the center point, and the horizontal axis of the image as the boundary... The axis, with the vertical axis of the image as... The axes provide the coordinates of each pixel in the monitored image.

[0021] The crane was in The first time during the next work session The surveillance image was mapped to the first [image] during this operation. Within the surveillance image, the mapped first... The angle between the two crane booms in the surveillance image is denoted as the angle between the two booms of the crane at the 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10 ... The first time during the next work session The working range within the Zhang surveillance image;

[0022] Obtain the crane in the The specific formula for calculating the probability of a lifting motion during the next operation is as follows:

[0023]

[0024] In the formula, Indicates the crane is in the The possibility of performing a hoisting motion during the next operation. Indicates the crane is in the The first time during the next work session The pixels of the crane boom in the surveillance image are... Maximum value on the axis Indicates the crane is in the The first time during the next work session The pixels of the crane boom in the surveillance image are... Maximum value on the axis Indicates the crane is in the The first time during the next work session The working range within the Zhang surveillance image, Indicates the crane is in the The number of surveillance images during each operation Represents the absolute value function. This represents the sigmoid function.

[0025] Furthermore, the specific method for obtaining continuous frame monitoring images of the crane's lifting motion includes:

[0026] Preset probability threshold ,like The crane was in During the next operation, the crane performs a lifting motion. The corresponding series of monitoring images for each operation are the continuous frame monitoring images of the crane performing the lifting motion.

[0027] Furthermore, the method for obtaining the corrected distance between each crane boom pixel and each pixel on the power transmission line in each frame of the monitoring image based on the distance between each crane boom pixel and each pixel on the power transmission line in the monitoring image and the degree of depth distortion of the plane containing each pixel on the power transmission line includes the following specific methods:

[0028]

[0029] In the formula, Indicates the first The first frame of the surveillance image The first crane boom pixel and the first The first on the power transmission line Correction distance per pixel Indicates the first The first on the power transmission line The degree of depth distortion in the plane containing each pixel. Indicates the first The first one in the surveillance image The first crane boom pixel and the first The first on the power transmission line The distance of one pixel.

[0030] Furthermore, the specific method for obtaining the final distance between the crane boom pixel and the power transmission line in each frame of the monitoring image within each lifting movement, based on the corrected distance sequence between different crane boom pixels and the same pixel on the power transmission line within the continuous frame monitoring images of the crane lifting movement, includes the following:

[0031] Within each frame of the monitoring image The crane boom pixel with the largest value on the axis is recorded as the top pixel of the boom; the pixel within each frame of the monitoring image is... The crane boom pixel with the smallest value on the axis is denoted as the end pixel of the boom.

[0032] Based on the acquisition time corresponding to each frame of the monitoring image, the crane is in the [number]th [frame]. The pixel at the top of the crane boom in all the surveillance images during the second lifting motion is compared with the first... The first on the power transmission line Arrange the correction distances of each pixel in ascending order to obtain the top correction distance sequence;

[0033] Based on the acquisition time corresponding to each frame of the monitoring image, the crane is in the [number]th [frame]. The pixels at the tail end of the crane boom in all the surveillance images during the second lifting motion are compared with the first... The first on the power transmission line Arrange the correction distances of each pixel from smallest to largest to obtain the tail correction distance sequence;

[0034] Using the top pixel of the crane boom and the first The first on the power transmission line The correction distance of each pixel is taken as one dimension, with the distance between the end pixel of the crane boom and the first pixel being the corrected distance of each pixel. The first on the power transmission line The corrected distance of each pixel is used as one dimension to construct a two-dimensional coordinate system;

[0035] Let the tail-corrected distance sequence and the apex-corrected distance sequence be the first... The values ​​of the i-th element are respectively the i-th element. The coordinates of each element on two corresponding axes are used to obtain several data points in a two-dimensional coordinate system.

[0036] The least squares method is used to fit all data points in the two-dimensional coordinate system, and the fitting error for each data point is obtained; the mean of the fitting errors for all data points is denoted as the nth... The second lifting of the internal crane was the first The first on the power transmission line Distortion interference degree of the plane containing each pixel;

[0037] According to the The second lifting of the internal crane was the first The distortion interference degree of the plane where each pixel point on the power transmission line is located is obtained, and the influence factor of the width and height distortion of the plane on the crane during each lifting movement is obtained. Then, the final distance between the crane boom pixel point and the power transmission line in each frame of the monitoring image within each lifting movement is obtained.

[0038] Furthermore, the statement based on the first The second lifting of the internal crane was the first The distortion interference degree of the plane where each pixel point on the transmission line is located is used to obtain the plane where the crane is located during each lifting movement and the influence factor of the width and height distortion of the plane on the crane during each lifting movement. The specific methods include:

[0039] The first The second lifting of the internal crane was the first The plane corresponding to the minimum distortion interference value of the plane containing all pixels on the i-th transmission line is denoted as the i-th plane. The plane in which the crane is located during the next lifting motion;

[0040] The normalized result of the minimum value is denoted as the i-th During the second lifting motion, the crane is affected by the width and height distortion factors of the plane in which it is located.

[0041] Furthermore, the specific method for obtaining the final distance between the crane boom pixel and the power transmission line in each frame of the monitoring image during each lifting movement includes:

[0042] The first The second lifting of the internal crane was the first The pixel corresponding to the minimum distortion interference value in the plane containing all pixels on the transmission line is denoted as the i-th pixel. On the first transmission line Pixels on the same plane of the crane during the second lifting motion;

[0043] Get the The second lifting movement The first one in the surveillance image The first crane boom pixel and the first The specific formula for calculating the final distance of each transmission line is as follows:

[0044]

[0045] In the formula, Indicates the first The second lifting movement The first one in the surveillance image The first crane boom pixel and the first The final distance of each transmission line, Indicates the first The second lifting movement The first one in the surveillance image The first crane boom pixel and the first On the first transmission line Correction distance of pixels on the same plane of the crane during the second lifting motion. Indicates the first During the second lifting motion, the crane is affected by the width and height distortion factors of the plane in which it is located.

[0046] The present invention also proposes a foreign object target sensing system around a power transmission line, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above method.

[0047] The beneficial effects of the technical solution of this invention are as follows: This invention obtains the distance between the crane boom and the power transmission line at the construction site by video recording and monitoring the power transmission line near the construction site. Based on the characteristics that the distance between each power transmission line and other power transmission lines is fixed and parallel in the actual scene, the depth distortion degree of the plane containing each power transmission line pixel in the monitoring image is obtained. This corrects the distance between each crane boom pixel and the power transmission line pixel, removing the interference of depth distortion information on the distance between the crane boom pixel and the power transmission line pixel. Since the length of the crane boom does not change during lifting operations, in reality, different pixels on the crane boom and the same power transmission line pixel... The distances between points are strongly correlated. However, due to the distortion of near-large and distant-small pixels, the position of the crane boom in different monitoring images during visual monitoring of power transmission lines weakens the correlation between the distances between different pixels on the crane boom and pixels on the same power transmission line, leading to increased fitting errors. By fitting the distances between different crane boom pixels and pixels on the same power transmission line in consecutive frame monitoring images, the fitting error is quantified to assess the impact of width and height distortion information on the distance between the crane boom and the power transmission line during monitoring. This reduces the interference of near-large and distant-small distortion on the distance between the crane boom and the power transmission line in the monitoring images, thereby improving the accuracy of foreign object detection around power transmission lines. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating the steps of a foreign object detection method for the vicinity of a power transmission line according to the present invention.

[0050] Figure 2 These are monitoring images of power transmission lines. Detailed Implementation

[0051] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a foreign object detection method and system for the vicinity of transmission lines proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0053] The following description, in conjunction with the accompanying drawings, details a specific scheme for a method and system for detecting foreign objects around power transmission lines provided by the present invention.

[0054] Please see Figure 1 The diagram illustrates a flowchart of a method for detecting foreign objects around a power transmission line according to an embodiment of the present invention. The method includes the following steps:

[0055] Step S001: Obtain the fixed distance of different power transmission lines and the power transmission lines and cranes in continuous frame monitoring images of the power transmission lines.

[0056] Specifically, surveillance cameras installed on transmission towers near the construction site are used to monitor the power transmission lines in the vicinity, obtaining video footage of the transmission lines. A video converter is then used to transform each second of the transmission line monitoring video into... Frame images are used to obtain continuous frame monitoring images of the transmission line. The preset number of frames in this embodiment... This example is used for illustration; other values ​​can be set in other embodiments. Converting video into multiple frames is a well-known technique and is not limited to this embodiment. The monitoring images of the transmission line are as follows: Figure 2 As shown.

[0057] Furthermore, semantic segmentation is used to obtain all power transmission lines and crane booms within each monitoring image. Semantic segmentation is a well-known technique in neural networks, which involves labeling power transmission lines and crane booms at construction sites in a large number of monitoring images of power transmission lines, training a neural network to build a semantic segmentation model, and using the cross-entropy loss function; simultaneously, the fixed distances between power transmission lines are obtained from the power grid system's backend database.

[0058] Step S002: Based on the distribution of pixels on different transmission lines in each frame of the monitoring image and the fixed distance between each transmission line and other transmission lines, obtain the depth distortion degree of the plane where each pixel on the transmission line is located.

[0059] It should be noted that when using cameras to capture images, there is a problem of converting three-dimensional information into two-dimensional planar information. This results in the presence of information from multiple planes within the monitoring image. Specifically, when using monitoring equipment on transmission towers to capture images of transmission lines, depth information is lost. Furthermore, the distance between different planes and the camera within the monitoring image varies, causing differences in the perspective effect of objects appearing larger when closer and smaller when farther away. Therefore, the monitoring image of the transmission line is divided into multiple planes, and the depth distortion information of each plane is obtained.

[0060] It should be further explained that, in reality, power transmission lines span multiple planes, meaning they possess depth information. Furthermore, two power lines are approximately parallel, ensuring that a pixel on one power line in a monitoring image shares the same plane as the pixel on another power line with the shortest distance between them. Therefore, based on this characteristic, the image is divided into multiple planes using the power lines. Since the distance between the two power lines on different planes is fixed when construction workers assume a power line exists between two towers, the depth distortion information of the plane containing each pixel on the power line is obtained based on the distance between the two power lines in different planes.

[0061] Specifically, the first On the power transmission line and the first The first on the power transmission line The pixel closest to the nth pixel is used as the nth pixel. The first power transmission line The pixel at the th point The corresponding pixels of each power transmission line.

[0062] Furthermore, the specific calculation formula for obtaining the depth distortion degree of the plane where each pixel on the transmission line is located is as follows:

[0063]

[0064] In the formula, Indicates the first The first on the power transmission line The degree of depth distortion in the plane containing each pixel. Indicates the first The first on the power transmission line The pixel and its position in the first... The distance between each transmission line and the corresponding pixel. Indicates the first The power transmission line and the first A fixed distance between transmission lines;

[0065] It should be noted that, The value represents the first The first on the power transmission line How many pixels in real life are equivalent to a unit pixel in the plane containing a pixel? The larger the value, the more significant the first... The first on the power transmission line The greater the distance between the plane containing the first pixel and the plane containing the camera, the better the pixel's performance. The first on the power transmission line The plane containing each pixel is more significantly affected by features that appear larger when closer and smaller when farther away.

[0066] This gives us the depth distortion of the plane containing each pixel on the transmission line.

[0067] Step S003: Based on the changes in the crane boom pixels in adjacent frame monitoring images, the probability of the crane performing a lifting motion during each operation is obtained, thereby obtaining continuous frame monitoring images of the crane performing a lifting motion.

[0068] It should be noted that in reality, when using a camera to photograph objects, two objects with the same height and width but different distances from the camera will appear to be different in size in the image. This means that during the image capture process, there is not only a depth distortion, but also a height distortion and a width distortion. Therefore, it is necessary to calculate the height distortion and width distortion within the surveillance image.

[0069] It's important to further clarify that when obtaining height and width distortion within a monitoring image by measuring the distance between the crane boom and the power line across multiple frames, it's crucial to ensure that the crane boom's movement within these frames is unaffected by depth distortion. Crane operations on construction sites primarily fall into two categories: one involves the crane boom gradually lifting objects off the ground, during which the plane on which the crane boom lies remains constant, meaning the depth distortion experienced by the crane boom is uniform across multiple frames of a single lifting motion; the other involves the crane boom rotating, during which the height of the crane boom above the ground remains constant, but the plane it traverses continuously changes, resulting in varying degrees of depth distortion across multiple frames of a single rotating crane motion. Therefore, by analyzing the position of the crane boom within multiple monitoring images during lifting motion, the degree of height and width distortion within the monitoring image can be obtained. Thus, acquiring multiple frames of monitoring images during the crane boom's lifting motion is essential.

[0070] It should be further explained that when determining the motion of a crane boom during operation, without considering distortion, when the crane boom rotates, the height of the crane boom above the ground remains constant, so ideally, the value of the top pixel on the vertical axis is fixed. However, when capturing an image, the height, position, and size of the object in the image are related to the distance between the object and the monitor. This causes the vertical axis height of the top pixel of the rotating crane boom within the monitored image to change, making it impossible to determine the type of motion of the crane boom based solely on whether the value of the top pixel on the vertical axis changes across multiple frames.

[0071] It's important to further clarify that when determining the movement of the crane boom during operation, the height of the crane boom above the ground changes during lifting motion, while it remains constant during rotational motion. The difference in height in the image is due to distortion. Because the monitoring equipment on the power transmission tower is far from the crane boom during filming, the lifting motion has a much greater impact on the vertical axis value of the top pixel of the crane boom than the distortion does during rotational motion. Therefore, by observing the changes in the vertical axis value of the top pixel of the crane boom across multiple frames, we can determine whether the crane boom is performing a lifting or rotational motion.

[0072] Specifically, all monitoring images are arranged in ascending order of their corresponding time to obtain a monitoring image sequence. The frame difference method is used to compare each monitoring image in the sequence with the crane boom in the next monitoring image. If the crane boom pixels in any two adjacent monitoring images are completely one-to-one corresponding, then both adjacent monitoring images are recorded as crane stop images. Within the monitoring image sequence, using the crane stop images as segmentation points, all monitoring images are divided into multiple image segments. All monitoring images within each image segment are considered as all monitoring images for one crane operation. Note that all monitoring images for one crane operation do not include crane stop images.

[0073] Furthermore, using the pixel at the bottom left corner of the monitored image as the center point, and the horizontal axis of the image as the focal point... The axis, with the vertical axis of the image as... The coordinates of each pixel in the monitored image are obtained by using the axis. The crane is positioned on the [axis name missing]. The first time during the next work session The surveillance image was mapped to the first [image] during this operation. Within the surveillance image, the mapped first... The angle between the two crane booms in the surveillance image is denoted as the angle between the two booms of the crane at the 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10 ... The first time during the next work session The working range within the Zhang surveillance image.

[0074] Furthermore, the crane was obtained in the first... The specific formula for calculating the probability of a lifting motion during the next operation is as follows:

[0075]

[0076] In the formula, Indicates the crane is in the The possibility of performing a hoisting motion during the next operation. Indicates the crane is in the The first time during the next work session The pixels of the crane boom in the surveillance image are... Maximum value on the axis Indicates the crane is in the The first time during the next work session The pixels of the crane boom in the surveillance image are... Maximum value on the axis Indicates the crane is in the The first time during the next work session The working range within the Zhang surveillance image, Indicates the crane is in the The number of surveillance images during each operation Represents the absolute value function. This represents the sigmoid function, which is used for normalization in this embodiment.

[0077] It should be noted that, Divide by This is to avoid the lifting motion being affected when the length of the crane boom performing the rotational motion is much greater than the length of the crane boom performing the lifting motion. Smaller than rotational motion The distance the boom moves during the crane's movement is determined by the distance the crane moves in the two images. Divide by This is to illustrate the change in the height of the crane boom tip from the ground when the crane's angle changes by a unit, thus avoiding the problem of the crane's rotation rate being different at different times.

[0078] Furthermore, a preset probability threshold is set. ,like The crane was in During the next operation, the crane performs a lifting motion. The consecutive frame monitoring images corresponding to each operation are consecutive frame monitoring images of the crane performing lifting movements. This embodiment has a preset probability threshold. This example is used for illustration; other values ​​can be set in other implementations.

[0079] Step S004: Based on the distance between each crane boom pixel and each pixel on the power transmission line in the monitoring image and the depth distortion of the plane where each pixel on the power transmission line is located, the corrected distance between each crane boom pixel and each pixel on the power transmission line in each frame of the monitoring image is obtained; based on the sequence of corrected distances between different crane boom pixels and the same pixel on the power transmission line in the continuous frame monitoring images of the crane lifting movement, the final distance between the crane boom pixels and the power transmission line in each frame of the monitoring image in each lifting movement is obtained, and this is used to detect foreign objects around the power transmission line.

[0080] It should be noted that when the crane boom is lifting, its length remains constant. This makes the lifting motion similar to a circular motion centered on the boom's tail end with a radius equal to the boom's length. Consequently, the position of each pixel on the crane boom is strongly correlated with the positions of other pixels on the same boom. Since the position of the power transmission line is fixed when photographing it, and distortion is not considered, the distances between different pixels on the crane boom and pixels on the same power transmission line are strongly correlated during the lifting motion. Therefore, based on the sequence of distances between different pixels on the crane boom and pixels on the power transmission line during the lifting motion, the effects of height and width distortion on the crane boom during this motion can be determined.

[0081] It should be further explained that when acquiring the impact of height and width distortion on the crane boom during each operation, if the image is unaffected by distortion, the distance sequences between different pixels on the crane boom and pixels on the same power line show a strong correlation, resulting in a small fitting error when fitting the data. However, in real-world photography, the image is affected by near-to-far distortion, which weakens the correlation between the distance sequences between different pixels on the crane boom and pixels on the same power line, increasing the fitting error. Therefore, the degree of height and width distortion in the monitored image is obtained by fitting the distances between different pixels on the crane boom and pixels on the same power line during the crane boom's lifting motion, based on the fitting error.

[0082] It's important to further explain that when photographing an object, there's a problem of converting three-dimensional information into two-dimensional planar information, meaning information from multiple planes is mapped onto a single plane. Therefore, when calculating the distance between the crane boom and the power line, it's first necessary to determine which power line pixel the crane boom lies within, and then calculate the distance between the crane boom and the power line pixels within that plane.

[0083] It should be further explained that since the plane on which the crane boom lies does not change when the crane boom is in lifting motion, the fitting error when calculating the correlation between the distances of two crane boom pixels and the transmission line pixels in the same plane is only affected by the degree of height and width distortion. However, when calculating the correlation between the distances of two crane boom pixels and the transmission line pixels not in the same plane, the fitting error is affected not only by the degree of height and width distortion but also by other factors. Therefore, when fitting the distances between two different pixels on the crane boom and a transmission line pixel, the fitting error after fitting the distance between two different pixels on the crane boom and the transmission line pixels in the same plane is smaller than the fitting error after fitting the distance between the transmission line pixels not in the same plane, thus revealing the plane on which the crane boom lies.

[0084] It should be further explained that when obtaining the plane containing the crane boom through fitting error, since pixels on a single power line lie in different planes, and the distortion levels of the planes containing pixels on different power lines vary, the distance between a pixel on the crane boom and pixels on different power lines is affected differently by depth distortion information. Therefore, when fitting the distances between different pixels on the crane boom and the same power line pixel, the distances between pixels on the boom and pixels on the power line are first corrected according to the distortion level of the plane containing each pixel on the power line. The two distance sequences are then fitted, and the fitting error is used to obtain the distance between the plane containing the crane boom and the plane affected by depth distortion. The distortion of the axis and the degree of distortion affect the distance in the image.

[0085] Specifically, obtain the pixel point of each crane boom and the first... The first on the power transmission line The formula for calculating the corrected distance of each pixel is as follows:

[0086]

[0087] In the formula, Indicates the first The first frame of the surveillance image The first crane boom pixel and the first The first on the power transmission line Correction distance per pixel Indicates the first The first on the power transmission line The degree of depth distortion in the plane containing each pixel. Indicates the first The first one in the surveillance image The first crane boom pixel and the first The first on the power transmission line The distance of one pixel.

[0088] It should be noted that, through and Multiplication solves the problem of the plane containing the crane boom pixels being affected by the near-large and far-small features.

[0089] Furthermore, within each frame of the monitoring image The crane boom pixel with the largest value on the axis is recorded as the top pixel of the boom; the pixel within each frame of the monitoring image is... The crane boom pixel with the smallest value on the axis is denoted as the boom end pixel. If a monitoring image contains... If there are multiple crane boom pixels with the largest values ​​on the axis, then the monitoring image will contain [the largest number of these pixels]. Among the multiple crane boom pixels with the largest values ​​on the axis The crane boom pixel with the largest value on the axis is denoted as the top pixel of the boom. If a monitoring image contains... If there are multiple crane boom pixels with the smallest value on the axis, then the monitoring image will contain [the smallest value]. Among the multiple crane boom pixels with the smallest values ​​on the axis The crane boom pixel with the largest value on the axis is denoted as the end pixel of the boom.

[0090] Furthermore, based on the acquisition time corresponding to each frame of the monitoring image, the crane is positioned in the [number]th [frame]. The pixel at the top of the crane boom in all the surveillance images during the second lifting motion is compared with the first... The first on the power transmission line Arrange the corrected distances of each pixel in ascending order to obtain the top corrected distance sequence. Based on the acquisition time corresponding to each frame of the monitoring image, the crane's position in the [frame name missing] frame is determined. The pixels at the tail end of the crane boom in all the surveillance images during the second lifting motion are compared with the first... The first on the power transmission line The correction distances of each pixel are arranged in ascending order to obtain the tail correction distance sequence.

[0091] Furthermore, the pixel at the top of the crane boom and the first... The first on the power transmission line The correction distance of each pixel is taken as one dimension, with the distance between the end pixel of the crane boom and the first pixel being the corrected distance of each pixel. The first on the power transmission line Construct a two-dimensional coordinate system with the corrected distance of each pixel as one dimension. Let the corrected distance sequence at the end and the corrected distance sequence at the top be the first pixel. The values ​​of the i-th element are respectively the i-th element. The coordinates of each element on two corresponding axes are used to obtain several data points in a two-dimensional coordinate system. The least squares method is then used to fit all data points in the two-dimensional coordinate system, and the fitting error for each data point is obtained. The least squares method is a well-known existing technique and will not be described in detail in this embodiment.

[0092] Furthermore, the mean of the fitting error for all data points is denoted as the th... The second lifting of the internal crane was the first The first on the power transmission line The distortion interference degree of the plane containing each pixel.

[0093] Furthermore, the first The second lifting of the internal crane was the first The plane corresponding to the minimum distortion interference degree of the plane containing all pixels on the i-th transmission line is denoted as the i-th plane. The plane where the crane is located during the second lifting movement. The second lifting of the internal crane was the first The pixel corresponding to the minimum distortion interference value in the plane containing all pixels on the transmission line is denoted as the i-th pixel. On the first transmission line The same plane pixels of the crane during the lifting motion.

[0094] Furthermore, to obtain the first The specific calculation formula for the influence factor of the width and height distortion of the plane on the crane during the second lifting motion is as follows:

[0095]

[0096] In the formula, Indicates the first During the second lifting motion, the crane is affected by the width and height distortion factors of the plane it is on. Indicates the first The second lifting of the internal crane was the first The minimum distortion interference value of the plane containing all pixels on a transmission line. This represents the sigmoid function.

[0097] It should be noted that, The larger the value, the more effective the monitoring is in the process of... When taking photos of the crane during a lifting operation, the distortion of the distance between the crane boom and the power transmission line due to width and height is more severe.

[0098] Furthermore, to obtain the first The second lifting movement The first frame in the surveillance image The first crane boom pixel and the first The specific formula for calculating the final distance of each transmission line is as follows:

[0099]

[0100] In the formula, Indicates the first The second lifting movement The first one in the surveillance image The first crane boom pixel and the first The final distance of each transmission line, Indicates the first The second lifting movement The first one in the surveillance image The first crane boom pixel and the first On the first transmission line Correction distance of pixels on the same plane of the crane during the second lifting motion. Indicates the first During the second lifting motion, the crane is affected by the width and height distortion factors of the plane in which it is located.

[0101] Furthermore, if the crane is continuously lifting... The minimum final distance between all crane boom pixels and the power transmission line in the frame monitoring image. The results after function normalization are all less than the distance threshold. If the construction site is deemed to pose a significant risk of damaging the power transmission lines, an electrician will be dispatched to inspect the site. This embodiment uses a preset distance threshold. This example is used for illustration; other values ​​can be set in other implementations. The preset number of consecutive images in this embodiment... This example is used for illustration; other values ​​can be set in other implementations.

[0102] This concludes the embodiment.

[0103] Another embodiment of the present invention provides a foreign object detection system around a power transmission line. The system includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the above method steps S001 to S004.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for sensing foreign objects around power transmission lines, characterized in that, The method includes the following steps: Acquire fixed distances of different power transmission lines and power transmission lines and cranes within continuous frame monitoring images of power transmission lines; Based on the distribution of pixels on different transmission lines within each frame of the monitoring image and the fixed distance between each transmission line and other transmission lines, the depth distortion degree of the plane containing each pixel on the transmission line is obtained; the specific method includes: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] On the power transmission line and the first The first on the power transmission line The pixel closest to the nth pixel is used as the nth pixel. The first power transmission line The pixel at the th point The corresponding pixels of each transmission line; the specific calculation formula for obtaining the depth distortion degree of the plane where each pixel on the transmission line is located is as follows: ； In the formula, Indicates the first The first on the power transmission line The degree of depth distortion in the plane containing each pixel. Indicates the first The first on the power transmission line The pixel and its position in the first... The distance between each transmission line and the corresponding pixel. Indicates the first The power transmission line and the first A fixed distance between transmission lines; Based on the changes in the crane boom pixels in adjacent frames of monitoring images, the probability of the crane performing a lifting motion during each operation is obtained, thus generating continuous frame monitoring images of the crane performing lifting motions. The specific method includes: arranging all monitoring images in ascending order of their corresponding time to obtain a monitoring image sequence; using a frame difference method to compare each monitoring image in the sequence with the crane boom in the next monitoring image; if the crane boom pixels in any two adjacent monitoring images are completely one-to-one corresponding, these two adjacent monitoring images are recorded as crane stop images; dividing all frame monitoring images in the monitoring image sequence into multiple image segments, using the crane stop images as segmentation points; and using all frame monitoring images within each image segment as all frame monitoring images for one crane operation; using the pixel at the lower left corner of the monitoring image as the center point, and the horizontal axis of the image as the dividing line. The axis, with the vertical axis of the image as... The axis is used to obtain the coordinates of each pixel in the monitored image; the crane is positioned on the 1st axis. The first time during the next work session The surveillance image was mapped to the first [image] during this operation. Within the surveillance image, the mapped first... The angle between the two crane booms in the surveillance image is denoted as the angle between the two booms of the crane at the 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10 ... The first time during the next work session The working range within the monitored image; obtaining the crane's position in the... The specific formula for calculating the probability of a lifting motion during the next operation is as follows: ； In the formula, Indicates the crane is in the The possibility of performing a hoisting motion during the next operation. Indicates the crane is in the The first time during the next work session The pixels of the crane boom in the surveillance image are... Maximum value on the axis Indicates the crane is in the The first time during the next work session The pixels of the crane boom in the surveillance image are... Maximum value on the axis Indicates the crane is in the The first time during the next work session The working range within the Zhang surveillance image, Indicates the crane is in the The number of surveillance images during each operation Represents the absolute value function. Represents the sigmoid function; Based on the distance between each crane boom pixel and each pixel on the power transmission line in the monitoring image, and the degree of depth distortion of the plane where each pixel on the power transmission line is located, the corrected distance between each crane boom pixel and each pixel on the power transmission line in each frame of the monitoring image is obtained. Based on the sequence of corrected distances between different crane boom pixels and the same pixel on the power transmission line in consecutive frames of the monitoring image during the crane's lifting motion, the final distance between the crane boom pixels and the power transmission line in each frame of the monitoring image during each lifting motion is obtained, and this is used to detect foreign objects around the power transmission line.

2. The method for detecting foreign objects around a power transmission line according to claim 1, characterized in that, The specific method for obtaining continuous frame monitoring images of the crane lifting motion is as follows: Preset probability threshold ,like The crane was in During the next operation, the crane performs a lifting motion. The corresponding series of monitoring images for each operation are the continuous frame monitoring images of the crane performing the lifting motion.

3. The method for detecting foreign objects around a power transmission line according to claim 1, characterized in that, The method for obtaining the corrected distance between each crane boom pixel and each pixel on the power transmission line in each frame of the monitoring image, based on the distance between each crane boom pixel in the monitoring image and each pixel on the power transmission line, and the degree of depth distortion of the plane containing each pixel on the power transmission line, includes the following specific methods: ； In the formula, Indicates the first The first frame of the surveillance image The first crane boom pixel and the first The first on the power transmission line Correction distance per pixel Indicates the first The first on the power transmission line The degree of depth distortion in the plane containing each pixel. Indicates the first The first one in the surveillance image The first crane boom pixel and the first The first on the power transmission line The distance of one pixel.

4. The method for detecting foreign objects around a power transmission line according to claim 1, characterized in that, The method for obtaining the final distance between the crane boom pixel and the power transmission line in each frame of monitoring images within each lifting motion, based on the corrected distance sequence between different crane boom pixels and the same pixel on the power transmission line within consecutive frames of monitoring images during the crane's lifting motion, includes the following specific methods: Within each frame of the monitoring image The crane boom pixel with the largest value on the axis is denoted as the top pixel of the boom. Within each frame of the monitoring image The crane boom pixel with the smallest value on the axis is denoted as the end pixel of the boom. Based on the acquisition time corresponding to each frame of the monitoring image, the crane is in the [number]th [frame]. The pixel at the top of the crane boom in all the surveillance images during the second lifting motion is compared with the first... The first on the power transmission line Arrange the correction distances of each pixel in ascending order to obtain the top correction distance sequence; Based on the acquisition time corresponding to each frame of the monitoring image, the crane is in the [number]th [frame]. The pixels at the tail end of the crane boom in all the surveillance images during the second lifting motion are compared with the first... The first on the power transmission line Arrange the correction distances of each pixel from smallest to largest to obtain the tail correction distance sequence; Using the top pixel of the crane boom and the first The first on the power transmission line The correction distance of each pixel is taken as one dimension, with the distance between the end pixel of the crane boom and the first pixel being the corrected distance of each pixel. The first on the power transmission line The corrected distance of each pixel is used as one dimension to construct a two-dimensional coordinate system; Let the tail-corrected distance sequence and the apex-corrected distance sequence be the first... The values ​​of the i-th element are respectively the i-th element. The coordinates of each element on two corresponding axes are used to obtain several data points in a two-dimensional coordinate system. The least squares method is used to fit all data points in the two-dimensional coordinate system, and the fitting error for each data point is obtained; the mean of the fitting errors for all data points is denoted as the nth... The second lifting of the internal crane was the first The first on the power transmission line Distortion interference degree of the plane containing each pixel; According to the The second lifting of the internal crane was the first The distortion interference degree of the plane where each pixel point on the power transmission line is located is obtained, and the influence factor of the width and height distortion of the plane on the crane during each lifting movement is obtained. Then, the final distance between the crane boom pixel point and the power transmission line in each frame of the monitoring image within each lifting movement is obtained.

5. The method for sensing foreign objects around a power transmission line according to claim 4, characterized in that, According to the first The second lifting of the internal crane was the first The distortion interference degree of the plane where each pixel point on the transmission line is located is used to obtain the plane where the crane is located during each lifting movement and the influence factor of the width and height distortion of the plane on the crane during each lifting movement. The specific methods include: The first The second lifting of the internal crane was the first The plane corresponding to the minimum distortion interference value of the plane containing all pixels on the i-th transmission line is denoted as the i-th plane. The plane in which the crane is located during the next lifting motion; The normalized result of the minimum value is denoted as the i-th During the second lifting motion, the crane is affected by the width and height distortion factors of the plane in which it is located.

6. The method for detecting foreign objects around a power transmission line according to claim 4, characterized in that, The method for obtaining the final distance between the crane boom pixel and the power transmission line in each frame of the monitoring image during each lifting movement includes: The first The second lifting of the internal crane was the first The pixel corresponding to the minimum distortion interference value in the plane containing all pixels on the transmission line is denoted as the i-th pixel. On the first transmission line Pixels on the same plane of the crane during the second lifting motion; Get the The second lifting movement The first one in the surveillance image The first crane boom pixel and the first The specific formula for calculating the final distance of each transmission line is as follows: ； In the formula, Indicates the first The second lifting movement The first one in the surveillance image The first crane boom pixel and the first The final distance of each transmission line, Indicates the first The second lifting movement The first one in the surveillance image The first crane boom pixel and the first On the first transmission line Correction distance of pixels on the same plane of the crane during the second lifting motion. Indicates the first During the second lifting motion, the crane is affected by the width and height distortion factors of the plane in which it is located.

7. A foreign object detection system for the vicinity of a power transmission line, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the foreign object detection method around a power transmission line as described in any one of claims 1-6.

Images