Method and system for sensing foreign object target around power transmission line

Abstract

The invention relates to the technical field of image processing, in particular to a foreign object target sensing method and system around a power transmission line, and the method comprises the steps: obtaining the power transmission line and a crane in each frame of monitoring image, and obtaining the depth distortion degree of a plane where each pixel point on the power transmission line is located, according to the change of the crane suspension arm pixel points in the adjacent frames of monitoring images, all frames of monitoring images of the crane during each hoisting motion are obtained, and then the correction distance between each crane suspension arm pixel point in each frame of monitoring image and each pixel point on the power transmission line is obtained; the plane where the crane is located during each hoisting motion and the width and height distortion influence factors of the plane where the crane is located during each hoisting motion are obtained, and protection measures of the power transmission line are obtained. The invention aims to solve the problem that the distance between a power transmission line and a crane jib in an image is different from the actual distance due to the fact that an object in the image is influenced by a large distance and a small distance when a camera is used for photographing the image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for sensing foreign objects around a power transmission line. Background Art

[0002] The power transmission link has always been plagued by various security threats during the power transmission process. The damage to the transmission line is mainly caused by climate change, the growth of trees around the transmission line, and damage during the construction process. Since the possibility of damage to the transmission line during the construction process can be reduced through human adjustment, it is necessary to immediately sense and detect foreign objects around the transmission line within the construction range.

[0003] At present, in the process of sensing foreign objects around the power transmission lines within the construction scope, the transmission lines are filmed and monitored by cameras installed on high-voltage towers, and the possibility of damage caused by the crane arm to the power transmission lines is judged by the distance between the crane arm and the power transmission lines in the image. Due to the pixel distortion problem of larger pixels when the image is taken, the actual distance between the crane arm and the power transmission lines is not accurately reflected in the image, which in turn affects the accuracy of foreign object sensing around the power transmission lines. Summary of the invention

[0004] A method and system for sensing foreign objects around a power transmission line of the present invention adopts the following technical solutions: The present invention proposes a method for sensing foreign objects around a power transmission line, the method comprising the following steps: Take the fixed distances of different transmission lines and the continuous frames of the transmission lines to monitor the transmission lines and cranes in the image; According to the distribution of pixels on different transmission lines in each frame of monitoring image and the fixed distance between each transmission line and other transmission lines, the depth distortion degree of the plane where each pixel on the transmission line is located is obtained; According to the change of the pixel points of the crane arm in the adjacent frame monitoring images, the possibility of the crane performing the lifting movement at each operation is obtained, and then the continuous frame monitoring images of the crane performing the lifting movement are obtained; According to the distance between each crane arm pixel point in the monitoring image and each pixel point on the transmission line and the depth distortion degree of the plane where each pixel point on the transmission line is located, the corrected distance between each crane arm pixel point and each pixel point on the transmission line in each frame of the monitoring image is obtained; according to the corrected distance sequence between different crane arm pixel points and the same pixel point on the transmission line in the continuous frame monitoring image of the crane performing the lifting movement, the final distance between the crane arm pixel point and the transmission line in each frame of the monitoring image in each lifting movement is obtained, and this is used to detect foreign objects around the transmission line.

[0005] Furthermore, the depth distortion degree of the plane where each pixel point on the transmission line is located is obtained according to the distribution of pixel points on different transmission lines in each frame of monitoring image and the fixed distance between each transmission line and other transmission lines, including the specific method of: The first The transmission line The first The pixel closest to the pixel is taken as the The first transmission line The pixel at The corresponding pixel points of the transmission lines; The specific calculation formula for obtaining the depth distortion degree of each pixel point on the transmission line is as follows: In the formula, Indicates The first The depth distortion of the plane where the pixel point is located, Indicates The first The pixel and its The distance between the pixels corresponding to the transmission lines, Indicates The transmission line and The fixed distance between transmission lines.

[0006] Furthermore, the possibility of the crane performing a lifting movement each time it works is obtained according to the change of the crane arm pixel points in the adjacent frame monitoring images, and the specific method includes: Arrange all the surveillance images in ascending order according to the corresponding time of each surveillance image frame to obtain a surveillance image sequence; use the frame difference method to compare each surveillance image in the surveillance image sequence with the crane arm in the next surveillance image; if the crane arm pixels in any two adjacent surveillance images are completely one-to-one corresponding, the two adjacent surveillance images are recorded as crane stop motion images; In the monitoring image sequence, the crane stop motion image is used as a segmentation point, and all the frame monitoring images are divided into a plurality of image segments, and all the frame monitoring images in each image segment are used as all the frame monitoring images when the crane is working once; According to the distribution difference of the pixels of the crane arm in the continuous frame monitoring images of the crane at each operation, the possibility of the crane performing the lifting movement at each operation is obtained.

[0007] Furthermore, the possibility of the crane performing a lifting movement at each operation is obtained according to the distribution difference of the crane arm pixel points in the continuous frame monitoring images at each operation of the crane, and the specific method includes: The pixel point at the lower left corner of the monitoring image is the circle point, and the horizontal axis of the image is Axis, with the vertical axis of the image as Axis, get the coordinates of each pixel in the monitoring image; Place the crane at The first time working The monitoring image is mapped to the In the monitoring image, the mapped The included angle between the two crane arms in the surveillance image is recorded as the crane's The first time working The working range within the monitoring image; Get the crane in The specific calculation formula for the possibility of doing lifting movement during work is as follows: In the formula, Indicates that the crane is Possibility of lifting movements during work, Indicates that the crane is The first time working The pixel points of the crane arm in the surveillance image are The maximum value on the axis, Indicates that the crane is The first time working The pixel points of the crane arm in the surveillance image are The maximum value on the axis, Indicates that the crane is The first time working The working range within the monitoring image is Indicates that the crane is The number of monitoring images during each work, represents the absolute value function, Represents the sigmoid function.

[0008] Furthermore, the specific method of obtaining the continuous frame monitoring image of the crane performing the lifting motion includes: Preset likelihood threshold ,like , the crane is at The crane performs lifting movements during the first work. The several continuous frame monitoring images corresponding to the first operation are continuous frame monitoring images of the crane performing lifting motion.

[0009] Furthermore, the method of obtaining the corrected distance between each crane arm pixel and each pixel on the transmission line in each frame of the monitoring image according to the distance between each crane arm pixel and each pixel on the transmission line in the monitoring image and the depth distortion degree of the plane where each pixel on the transmission line is located includes the following specific methods: In the formula, Indicates The first The crane arm pixel and the The first The corrected distance of pixels, Indicates The first The depth distortion of the plane where the pixel point is located, Indicates The surveillance image The crane arm pixel and the The first The distance of pixels.

[0010] Furthermore, the method of obtaining the final distance between the crane arm pixel point and the transmission line in each frame of monitoring image in each lifting movement according to the corrected distance sequence between different crane arm pixel points and the same pixel point on the transmission line in the continuous frame monitoring image of the crane lifting movement includes the following specific methods: Each frame of monitoring image The crane arm pixel with the largest value on the axis is recorded as the top pixel of the arm; The crane arm pixel with the smallest value on the axis is recorded as the arm end pixel; According to the acquisition time corresponding to each frame of monitoring image, the crane is placed in the first The pixel points at the top of the crane arm in all surveillance images during the first lifting movement are The first The correction distances of the pixels are arranged from small to large to obtain the top correction distance sequence; According to the acquisition time corresponding to each frame of monitoring image, the crane is placed in the first The pixels of the crane boom tail end in all surveillance images during the first lifting movement are consistent with the pixels of the first The first The correction distances of the pixels are arranged from small to large to obtain the tail correction distance sequence; The pixel point at the top of the crane arm and the The first The correction distance of pixels is one dimension, with the pixel point at the end of the crane arm and the pixel point at the end of the crane arm being the same as the pixel point at the end of the crane arm. The first The corrected distance of each pixel point is taken as one dimension to construct a two-dimensional coordinate system; Let the tail end correction distance sequence and the top correction distance sequence The values ​​of the elements are The coordinates of an element on two corresponding axes are used to obtain several data points in a two-dimensional coordinate system; Use the least squares method to fit all data points in the two-dimensional coordinate system and obtain the fitting error of each data point. The mean of the fitting errors of all data points is recorded as The crane is in the first lifting movement The first The distortion interference degree of the plane where the pixel points are located; According to The crane is in the first lifting movement The distortion interference degree of the plane where each pixel point on the transmission line is located is calculated, and the plane where the crane is located during each lifting movement and the influence factors of the width and height distortion of the plane where the crane is located during each lifting movement are obtained; then the final distance between the crane arm pixel point and the transmission line in each frame of the monitoring image within each lifting movement is obtained.

[0011] Furthermore, according to The crane is in the first lifting movement The distortion interference degree of the plane where each pixel point on the transmission line is located is obtained, and the plane where the crane is located during each lifting movement and the influence factor of the width and height distortion of the plane where the crane is located during each lifting movement are obtained. The specific method includes: The first The crane is in the first lifting movement The plane corresponding to the minimum value of the distortion interference of all pixels on the transmission line is denoted as The plane where the crane is located during the first lifting movement; The result after normalization of the minimum value is recorded as The factor affecting the width and height distortion of the plane on which the crane is located during the first lifting movement.

[0012] Furthermore, the method of obtaining the final distance between the crane arm pixel point and the transmission line in each frame of the monitoring image during each lifting movement includes: The first The crane is in the first lifting movement The pixel point corresponding to the minimum value of the distortion interference degree of all pixels on the transmission line is recorded as Transmission line No. The pixels on the same plane as the crane during the first lifting movement; Get the The first lifting movement The surveillance image The crane arm pixel and the The specific calculation formula for the final distance of a transmission line is as follows: In the formula, Indicates The first lifting movement The surveillance image The crane arm pixel and the The final distance of the transmission line, Indicates The first lifting movement The surveillance image The crane arm pixel and the Transmission line No. Corrected distance of the crane's pixels on the same plane during the first lifting movement, Indicates The factor affecting the width and height distortion of the plane on which the crane is located during the first lifting movement.

[0013] The present invention also proposes a foreign object target perception system around a transmission line, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above method.

[0014] The beneficial effects of the technical solution of the present invention are as follows: the present invention obtains the distance between the crane boom and the transmission line at the construction site by performing video shooting and monitoring of the transmission line near the construction site. Based on the characteristics that the distance between each transmission line and other transmission lines in the actual scene is fixed and the transmission lines are parallel, the depth distortion degree of the plane where each transmission line pixel point in the monitoring image is located is obtained, and the distance between each crane boom pixel point and the transmission line pixel point is corrected to remove the interference of the depth distortion information on the distance between the crane boom pixel point and the transmission line pixel point. Since the length of the crane boom does not change when the crane boom is performing the lifting movement, in actual situations, different pixel points on the crane boom and the same transmission line pixel points are not necessarily the same. The distances between the points have a strong correlation. Since the position of the crane arm in different monitoring images is affected by the distortion of near large and far small during visual monitoring of the transmission line, the correlation between the distances between different pixel points on the crane arm in the monitoring image and the pixel points on the same transmission line is weakened, resulting in an increase in the fitting error. The fitting error after fitting the distances between different crane arm pixels in continuous frame monitoring images and the pixel points on the same transmission line is quantified to quantify the influence of the width and height distortion information on the distance between the crane arm and the transmission line during the monitoring process, thereby reducing the interference of the distortion of near large and far small on the distance between the crane arm and the transmission line in the monitoring image, thereby improving the accuracy of foreign object perception and detection around the transmission line. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A flowchart of the steps of a method for sensing foreign objects around a power transmission line according to the present invention; Figure 2 This is a monitoring image of a power transmission line. DETAILED DESCRIPTION

[0017] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the foreign body target sensing method and system around the transmission line proposed by the present invention, its specific implementation method, structure, characteristics and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0018] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0019] The following is a detailed description of a method and system for sensing foreign bodies around a power transmission line provided by the present invention in conjunction with the accompanying drawings.

[0020] See also Figure 1 , which shows a flowchart of a method for sensing foreign objects around a power transmission line provided by an embodiment of the present invention, the method comprising the following steps: Step S001: Obtain fixed distances of different transmission lines and the transmission lines and cranes in continuous frame monitoring images of the transmission lines.

[0021] Specifically, the monitoring equipment installed on the transmission tower near the construction site is used to monitor the transmission line near the construction site to obtain the monitoring video of the transmission line. The monitoring video of the transmission line is converted into a video converter every 1 second. Frame images are obtained to obtain continuous frame monitoring images of the transmission line. The number of frames preset in this embodiment is , which is described as an example, and other values ​​may be set in other implementations. Converting the video into multiple frames of images is a known technology, and this embodiment does not limit it. The monitoring image of the power transmission line is as follows Figure 2 shown.

[0022] Furthermore, all the transmission lines and crane arms in each monitoring image are obtained through semantic segmentation. Semantic segmentation is a well-known technology in neural networks, that is, the transmission lines and crane arms at the construction site in the monitoring images of a large number of transmission lines are labeled, and a semantic segmentation model is constructed by training the neural network, and the loss function adopts the cross entropy loss function; at the same time, the fixed distance between the transmission lines in the background database of the power grid system is obtained.

[0023] Step S002: According to the distribution of pixels on different transmission lines in each frame of monitoring image and the fixed distance between each transmission line and other transmission lines, the depth distortion degree of the plane where each pixel on the transmission line is located is obtained.

[0024] It should be noted that when using a camera to take pictures of an image, there is a problem of converting three-dimensional information into two-dimensional information, which results in the monitoring image containing information on multiple planes, that is, when using the monitoring on the transmission tower to shoot the transmission line, the depth information is lost. In addition, in the monitoring image, the distance between different planes and the camera is different, so that different planes in the monitoring image are affected differently by the near big and far small. Therefore, the monitoring image of the transmission line is divided into multiple planes, and the depth distortion information of each plane is obtained.

[0025] It should be further explained that in reality, the transmission lines span multiple planes, that is, the transmission lines have depth information, and the two transmission lines are approximately parallel, so that in the monitoring image, a pixel point on one transmission line and the pixel point with the shortest distance on the other transmission line are in the same plane. Therefore, according to this feature, the image is divided into multiple planes by the transmission line. When the construction workers assume the transmission line between the two transmission towers, the distance between the two transmission lines in different planes is fixed. Therefore, according to the distance between the two transmission lines in different planes, the depth distortion information of the plane where each pixel point on the transmission line is located is obtained.

[0026] Specifically, the The transmission line The first The pixel closest to the first pixel is The first transmission line The pixel at The corresponding pixel points of the transmission lines.

[0027] Furthermore, the specific calculation formula for obtaining the depth distortion degree of each pixel point on the transmission line is as follows: In the formula, Indicates The first The depth distortion of the plane where the pixel point is located, Indicates The first The pixel and its The distance between the pixels corresponding to the transmission lines, Indicates The transmission line and The fixed distance between transmission lines; It should be noted that The value of The first How many pixels in real life are equivalent to the unit pixel point on the plane where the pixels are located? The larger the value, the The first The farther the distance between the plane where the pixel is located and the plane where the camera is located, the The first The plane where the pixel is located is more affected by the features that are larger near and smaller far away.

[0028] At this point, the depth distortion degree of each pixel point on the transmission line is obtained.

[0029] Step S003: According to the change of the pixel points of the crane arm in the adjacent frame monitoring images, the possibility of the crane performing a lifting movement each time it works is obtained, and then the continuous frame monitoring images of the crane performing the lifting movement are obtained.

[0030] It should be noted that in reality, when using a camera to shoot an object, when two objects with the same height and width but different distances from the camera have different sizes in the image, that is, in the process of shooting the image, there is not only a depth distortion, but also a height distortion and a width distortion. Therefore, the height distortion and width distortion in the monitoring image are calculated.

[0031] It should be further explained that when obtaining the height distortion and width distortion in the monitoring image through the distance between the crane arm and the transmission line in multiple frames of images, it is necessary to ensure that the movement of the crane arm in the multiple frames of images is not affected by the depth distortion. There are two main types of crane work in the construction site. One is the lifting movement of the crane arm gradually lifting the object on the ground off the ground. In this process, the plane where the crane arm is located does not change, that is, the degree of depth distortion of the crane arm in the multiple frames of monitoring images of a lifting movement is the same; the other is the crane arm is rotating. In this process, the height of the crane arm from the ground does not change, and the plane passed by the crane arm is constantly changing, that is, the degree of depth distortion of the crane arm in the multiple frames of monitoring images of a crane arm rotating movement is different. Therefore, the position of the crane arm in multiple monitoring images during the lifting movement is used to obtain the degree of height and width distortion in the monitoring image. Therefore, multiple frames of monitoring images of the crane arm performing lifting movement are obtained.

[0032] It is further necessary to explain that when judging what kind of movement the crane arm is doing at work, without considering distortion, when the crane arm is rotating, the height of the crane arm from the ground does not change, so that the value of the top pixel point of the crane arm on the vertical axis is fixed under ideal conditions. However, when shooting an image, the height, position and size of the object in the image are related to the distance between the object and the monitor, so that the height of the vertical axis of the top pixel point of the crane arm rotating in the monitor image is also changing, making it impossible to judge what kind of movement the crane arm is doing based on whether the value of the top pixel point of the crane arm on the vertical axis changes in multiple frames of images.

[0033] It should be further explained that when judging what kind of movement the crane arm is doing at work, the height of the crane arm above the ground changes when the crane arm is doing lifting movement, and the height of the crane arm above the ground does not change when the crane arm is doing rotational movement. The change in the image is due to the influence of distortion. Since the distance between the monitoring on the transmission tower and the crane arm is far when shooting the crane arm, the lifting movement of the crane arm on the vertical axis affects the value of the pixel point at the top of the crane arm much more than the effect of the distortion relationship on the value of the pixel point at the top of the crane arm on the vertical axis when the crane arm is doing rotational movement. Therefore, according to the change of the value of the pixel point at the top of the crane arm on the vertical axis in multiple frames of images, it is judged whether the crane arm is doing lifting movement or rotational movement when the crane arm is moving.

[0034] Specifically, all monitoring image frames are arranged from small to large according to the corresponding time of each monitoring image frame to obtain a monitoring image sequence; each monitoring image frame in the monitoring image sequence is compared with the crane arm in the next monitoring image using the frame difference method. If the crane arm pixels in any two adjacent monitoring image frames are completely one-to-one corresponding, the two adjacent monitoring image frames are recorded as crane stop motion images. In the monitoring image sequence, all monitoring image frames are divided into multiple image segments using the crane stop motion image as a segmentation point, and all monitoring image frames in each image segment are used as all monitoring images of the crane when it is working once. Among them, all monitoring images of the crane when it is working once do not include the crane stop motion image.

[0035] Furthermore, the pixel point at the lower left corner of the monitoring image is taken as the dot and the horizontal axis of the image is taken as Axis, with the vertical axis of the image as Axis, get the coordinates of each pixel in the monitoring image. The first time working The monitoring image is mapped to the In the monitoring image, the mapped The included angle between the two crane arms in the surveillance image is recorded as the crane's The first time working Monitor the working range within the image.

[0036] Further, obtain the crane in the The specific calculation formula for the possibility of doing lifting movement during work is as follows: In the formula, Indicates that the crane is Possibility of lifting movements during work, Indicates that the crane is The first time working The pixel points of the crane arm in the surveillance image are The maximum value on the axis, Indicates that the crane is The first time working The pixel points of the crane arm in the surveillance image are The maximum value on the axis, Indicates that the crane is The first time working The working range within the monitoring image is Indicates that the crane is The number of monitoring images during each work, represents the absolute value function, represents a sigmoid function, which is used for normalization processing in this embodiment.

[0037] It should be noted that Divide by This is to avoid the situation where the length of the crane arm doing the rotating motion is much longer than the length of the crane arm doing the lifting motion. Less than rotational motion ; The distance the boom moves when the crane moves in the two images; Divide by This is to illustrate the change in height of the top of the crane boom from the ground when the angle of the crane changes by a unit, thus avoiding the problem of different rotation rates of the crane at different times.

[0038] Furthermore, a preset possibility threshold ,like , the crane is at The crane performs lifting movements during the first work. The corresponding several continuous frames of monitoring images during the operation are continuous frames of monitoring images of the crane doing the lifting movement. , this is used as an example for description, and other values ​​may be set in other implementation modes.

[0039] Step S004: According to the distance between each crane arm pixel point and each pixel point on the transmission line in the monitoring image and the depth distortion degree of the plane where each pixel point on the transmission line is located, the corrected distance between each crane arm pixel point and each pixel point on the transmission line in each frame of the monitoring image is obtained; according to the corrected distance sequence between different crane arm pixel points and the same pixel point on the transmission line in the continuous frame monitoring image of the crane performing the lifting movement, the final distance between the crane arm pixel point and the transmission line in each frame of the monitoring image in each lifting movement is obtained, and the surrounding foreign objects of the transmission line are detected based on this.

[0040] It should be noted that when the crane arm is lifting, the length of the crane arm will not change, so that the lifting movement of the crane arm is similar to a circular motion with the tail end of the arm as the center and the length of the arm as the radius, so that the position of each pixel on the crane arm has a strong correlation with the position of other pixel points on the arm. Since the position of the transmission line is fixed when taking pictures of the transmission line, when the crane arm is lifting, the distance between different pixel points on the crane arm and the pixel points of the same transmission line has a strong correlation without considering the influence of distortion. Therefore, according to the distance sequence between different pixel points on the crane arm and the pixel points of the transmission line when the crane arm is lifting, the influence of the height and width distortion on the crane arm during this movement is obtained.

[0041] It should be further explained that when obtaining the influence of the height and width distortion of the crane arm during each operation of the crane, if the image is not affected by the distortion, the two distance sequences between different pixel points on the crane arm and the pixel points on the same transmission line have a strong correlation, so that the fitting error is small when fitting the data. However, when taking pictures of images in real life, the captured images are affected by the distortion of near large and far small, which weakens the correlation between the two distance sequences between different pixel points on the crane arm and the pixel points on the same transmission line, so that the fitting error increases when fitting the data. Therefore, according to the fitting error after fitting the distance between different pixel points on the crane arm and the pixel points on the same transmission line when the crane arm is lifting, the degree of distortion of the height and width in the monitoring image is obtained.

[0042] It should be further explained that when taking a photo of an object, there is a problem of converting three-dimensional information into two-dimensional information, that is, mapping information in multiple planes into one plane. Therefore, when calculating the distance between the crane boom and the power transmission line, it is first necessary to determine which plane the crane boom is in, and then calculate the distance between the crane boom and the power transmission line pixel point in the plane where it is located.

[0043] It should be further explained that, when the crane arm is lifting, the plane where the crane arm is located will not change, so when calculating the correlation between the distance between two crane arm pixels and the transmission line pixels in the same plane, the fitting error is only affected by the degree of distortion of height and width; and when calculating the correlation between the distance between two crane arm pixels and the transmission line pixels that are not in the same plane, the fitting error is not only affected by the degree of distortion of height and width, but also by other factors. So that when fitting the distance between two different pixels on the crane arm and a transmission line pixel, the fitting error after fitting the distance between two different pixels on the crane arm and the transmission line pixel in the same plane is less than the fitting error after fitting the distance between the transmission line pixels that are not in the same plane, and then the plane where the crane arm is located is obtained.

[0044] It should be further explained that when obtaining the plane where the crane arm is located through fitting errors, since the pixels on a transmission line are in different planes, and the distortion degrees of the planes where the pixels on different transmission lines are located are different, the distance between a pixel on the crane arm and the pixels of different transmission lines is affected by the depth distortion information to different degrees. Therefore, when fitting the distance between different pixels on the crane arm and the same transmission pixel, first correct the distance between the pixel on the arm and the pixel on the transmission line according to the degree of distortion of the plane where the pixel on each transmission line is located, fit the two distance sequences, and obtain the distance between the plane where the crane arm is located and the distance between the plane where the crane arm is affected by the crane arm according to the fitting error. The influence of the axis distortion and the degree of axis distortion is used to correct the distance in the image.

[0045] Specifically, obtain the pixel points of each crane arm and the The first The calculation formula for the corrected distance of a pixel is as follows: In the formula, Indicates The first The crane arm pixel and the The first The corrected distance of pixels, Indicates The first The depth distortion of the plane where the pixel point is located, Indicates The surveillance image The crane arm pixel and the The first The distance of pixels.

[0046] It should be noted that through and Multiplication solves the problem that the plane where the crane boom pixel is located is affected by features that are larger near and smaller far away.

[0047] Furthermore, each frame of the monitoring image The crane arm pixel with the largest value on the axis is recorded as the top pixel of the arm; The crane arm pixel with the smallest value on the axis is recorded as the end pixel of the arm. If there are multiple crane arm pixels with the largest value on the axis, Among the multiple crane arm pixels with the largest value on the axis The crane arm pixel with the largest value on the axis is recorded as the top pixel of the arm. If there are multiple crane arm pixels with the smallest value on the axis, Among the multiple crane arm pixels with the smallest value on the axis The crane arm pixel with the largest value on the axis is recorded as the arm end pixel.

[0048] Furthermore, the crane is placed at the first The pixel points at the top of the crane arm in all surveillance images during the first lifting movement are The first The correction distances of the pixels are arranged from small to large to obtain the top correction distance sequence. The crane is placed at the first The pixels of the crane boom tail end in all surveillance images during the first lifting movement are consistent with the pixels of the first The first The corrected distances of the pixels are arranged from small to large to obtain the tail corrected distance sequence.

[0049] Furthermore, the pixel point at the top of the crane arm is The first The correction distance of pixels is one dimension, with the pixel point at the end of the crane arm and the pixel point at the end of the crane arm being the same as the pixel point at the end of the crane arm. The first The correction distance of each pixel point is taken as one dimension to construct a two-dimensional coordinate system. The values ​​of the elements are The coordinates of the elements on the two corresponding axes are obtained to obtain a number of data points in the two-dimensional coordinate system. The least squares method is used to fit all the data points in the two-dimensional coordinate system to obtain the fitting error of each data point. Among them, the least squares method is a well-known technology and will not be described in detail in this embodiment.

[0050] Furthermore, the mean of the fitting errors of all data points is recorded as The crane is in the first lifting movement The first The distortion interference degree of the plane where the pixel points are located.

[0051] Furthermore, The crane is in the first lifting movement The plane corresponding to the minimum distortion interference of all pixels on the transmission line is denoted as The plane where the crane is located during the first lifting movement. The crane is in the first lifting movement The pixel point corresponding to the minimum value of the distortion interference degree of all pixels on the transmission line is recorded as Transmission line No. The pixels on the same plane as the crane during the first lifting movement.

[0052] Further, get the The specific calculation formula for the influence factor of the width and height distortion of the plane on which the crane is located during the first lifting movement is as follows: In the formula, Indicates The influence factor of the width and height distortion of the plane on which the crane is located during the first lifting movement, Indicates The crane is in the first lifting movement The minimum value of the distortion interference of the plane where all pixels on the transmission line are located, Represents the sigmoid function.

[0053] It should be noted that The larger the value is, the more When taking pictures of the crane during its first lifting operation, the distance between the crane boom and the transmission line is more severely distorted by the width and height.

[0054] Further, get the The first lifting movement The first frame in the monitoring image The crane arm pixel and the The specific calculation formula for the final distance of a transmission line is as follows: In the formula, Indicates The first lifting movement The surveillance image The crane arm pixel and the The final distance of the transmission line, Indicates The first lifting movement The surveillance image The crane arm pixel and the Transmission line No. Corrected distance of the crane's pixels on the same plane during the first lifting movement, Indicates The factor affecting the width and height distortion of the plane on which the crane is located during the first lifting movement.

[0055] Furthermore, if the crane is continuously The minimum value of the final distance between all crane arm pixels and the transmission line in the frame monitoring image is The results after function normalization are all less than the distance threshold , it is considered that the construction site is likely to cause damage to the transmission line, and electricians are dispatched to inspect the construction site. , which is described as an example, and other implementations may be set to other values. The number of continuous images preset in this embodiment , this is used as an example for description, and other values ​​may be set in other implementation modes.

[0056] At this point, this embodiment is completed.

[0057] Another embodiment of the present invention provides a foreign object target perception system around a transmission line, the system comprising a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, the above method steps S001 to S004 are implemented.

[0058] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for sensing foreign objects around a power transmission line, characterized in that: The method comprises the following steps: obtaining fixed distances of different transmission lines and transmission lines and cranes in continuous frame monitoring images of the transmission lines; obtaining the depth distortion degree of the plane where each pixel point on the transmission line is located according to the distribution of pixel points on different transmission lines in each frame monitoring image and the fixed distance between each transmission line and other transmission lines; obtaining the possibility of the crane performing a lifting movement at each operation according to the changes of the crane arm pixel points in adjacent frame monitoring images, and then obtaining continuous frame monitoring images of the crane performing the lifting movement; obtaining the corrected distance between each crane arm pixel point and each pixel point on the transmission line in each frame monitoring image according to the distance between each crane arm pixel point and each pixel point on the transmission line in the monitoring image and the depth distortion degree of the plane where each pixel point on the transmission line is located; obtaining the final distance between the crane arm pixel point and the transmission line in each frame monitoring image in each lifting movement according to the corrected distance sequence between different crane arm pixel points and the same pixel point on the transmission line in the continuous frame monitoring images of the crane performing the lifting movement, and performing peripheral foreign body detection on the transmission line accordingly.

2. A method for sensing foreign objects around a power transmission line according to claim 1, characterized in that: The method of obtaining the depth distortion degree of the plane where each pixel point on the transmission line is located according to the distribution of the pixel points on different transmission lines in each frame of the monitoring image and the fixed distance between each transmission line and other transmission lines includes the following specific methods: The first The transmission line The first The pixel closest to the pixel is taken as the The first transmission line The pixel at The corresponding pixel points of the transmission lines; The specific calculation formula for obtaining the depth distortion degree of each pixel point on the transmission line is as follows: In the formula, Indicates The first The depth distortion of the plane where the pixel point is located, Indicates The first The pixel and its The distance between the pixels corresponding to the transmission lines, Indicates The transmission line and The fixed distance between transmission lines.

3. The method for sensing foreign objects around a power transmission line according to claim 1, characterized in that: The possibility of the crane performing a lifting motion each time it works is obtained according to the change of the pixel points of the crane arm in the adjacent frame monitoring images, and the specific method includes: Arrange all the surveillance images in ascending order according to the corresponding time of each surveillance image frame to obtain a surveillance image sequence; use the frame difference method to compare each surveillance image in the surveillance image sequence with the crane arm in the next surveillance image; if the crane arm pixels in any two adjacent surveillance images are completely one-to-one corresponding, the two adjacent surveillance images are recorded as crane stop motion images; In the monitoring image sequence, the crane stop motion image is used as a segmentation point, and all the frame monitoring images are divided into a plurality of image segments, and all the frame monitoring images in each image segment are used as all the frame monitoring images when the crane is working once; According to the distribution difference of the pixels of the crane arm in the continuous frame monitoring images of the crane at each operation, the possibility of the crane performing the lifting movement at each operation is obtained.

4. A method for sensing foreign objects around a power transmission line according to claim 3, characterized in that: The possibility of the crane performing a lifting movement each time it works is obtained according to the distribution difference of the crane arm pixel points in the continuous frame monitoring images of the crane each time it works, and the specific method includes: The pixel point at the lower left corner of the monitoring image is the circle point, and the horizontal axis of the image is Axis, with the vertical axis of the image as Axis, get the coordinates of each pixel in the monitoring image; Place the crane at The first time working The monitoring image is mapped to the In the monitoring image, the mapped The included angle between the two crane arms in the surveillance image is recorded as the crane's The first time working The working range within the monitoring image; Get the crane in The specific calculation formula for the possibility of doing lifting movement during work is as follows: In the formula, Indicates that the crane is Possibility of lifting movements during work, Indicates that the crane is The first time working The pixels of the crane arm in the surveillance image are The maximum value on the axis, Indicates that the crane is The first time working The pixels of the crane arm in the surveillance image are The maximum value on the axis, Indicates that the crane is The working range within the monitoring image during the first work, Indicates that the crane is The number of monitoring images during each work, represents the absolute value function, Represents the sigmoid function.

5. A method for sensing foreign objects around a power transmission line according to claim 4, characterized in that: The specific method of obtaining the continuous frame monitoring image of the crane doing the lifting movement includes: Preset likelihood threshold , if the crane is in the The crane performs lifting movements during the first work. The several continuous frame monitoring images corresponding to the first operation are continuous frame monitoring images of the crane performing lifting motion.

6. The method for sensing foreign objects around a power transmission line according to claim 1, characterized in that: The method of obtaining the corrected distance between each crane arm pixel point and each pixel point on the transmission line in each frame of the monitoring image according to the distance between each crane arm pixel point and each pixel point on the transmission line in the monitoring image and the depth distortion degree of the plane where each pixel point on the transmission line is located includes the following specific methods: In the formula, Indicates The first The crane arm pixel and the The first The corrected distance of pixels, Indicates The first The depth distortion of the plane where the pixel point is located, Indicates The surveillance image The crane arm pixel and the The first The distance of pixels.

7. The method for sensing foreign objects around a power transmission line according to claim 1, characterized in that: The method of obtaining the final distance between the crane arm pixel point and the transmission line in each frame of monitoring image during each lifting movement according to the corrected distance sequence between different crane arm pixel points and the same pixel point on the transmission line in the continuous frame monitoring image of the crane performing the lifting movement includes the following specific methods: Each frame of monitoring image The crane arm pixel with the largest value on the axis is recorded as the arm top pixel; Each frame of monitoring image The crane arm pixel with the smallest value on the axis is recorded as the arm end pixel; According to the acquisition time corresponding to each frame of monitoring image, the crane is placed in the first The pixel points at the top of the crane arm in all surveillance images during the first lifting movement are The first The correction distances of the pixels are arranged from small to large to obtain the top correction distance sequence; According to the acquisition time corresponding to each frame of monitoring image, the crane is placed in the first The pixels of the crane boom tail end in all surveillance images during the first lifting movement are consistent with the pixels of the first The first The correction distances of the pixels are arranged from small to large to obtain the tail correction distance sequence; The pixel point at the top of the crane arm and the The first The correction distance of pixels is one dimension, with the pixel point at the end of the crane arm and the pixel point at the end of the crane arm being the same as the pixel point at the end of the crane arm. The first The corrected distance of each pixel point is taken as one dimension to construct a two-dimensional coordinate system; Let the tail end correction distance sequence and the top correction distance sequence The values ​​of the elements are The coordinates of an element on two corresponding axes are used to obtain several data points in a two-dimensional coordinate system; Use the least squares method to fit all data points in the two-dimensional coordinate system and obtain the fitting error of each data point. The mean of the fitting errors of all data points is recorded as The crane is in the first lifting movement The first The distortion interference degree of the plane where the pixel points are located; According to The crane is in the first lifting movement The distortion interference degree of the plane where each pixel point on the transmission line is located is calculated, and the plane where the crane is located during each lifting movement and the influence factors of the width and height distortion of the plane where the crane is located during each lifting movement are obtained; then the final distance between the crane arm pixel point and the transmission line in each frame of the monitoring image within each lifting movement is obtained.

8. A method for sensing foreign objects around a power transmission line according to claim 7, characterized in that: According to the The crane is in the first lifting movement The distortion interference degree of the plane where each pixel point on the transmission line is located is obtained, and the plane where the crane is located during each lifting movement and the influence factor of the width and height distortion of the plane where the crane is located during each lifting movement are obtained. The specific method includes: The first The crane is in the first lifting movement The plane corresponding to the minimum value of the distortion interference of all pixels on the transmission line is denoted as The plane where the crane is located during the first lifting movement; The result after normalization of the minimum value is recorded as The factor affecting the width and height distortion of the plane on which the crane is located during the first lifting movement.

9. The method for sensing foreign objects around a power transmission line according to claim 7, characterized in that: The method of obtaining the final distance between the crane arm pixel point and the transmission line in each frame of the monitoring image during each lifting movement includes the following specific methods: The first The crane is in the first lifting movement The pixel point corresponding to the minimum value of the distortion interference degree of all pixels on the transmission line is recorded as Transmission line No. The pixel points on the same plane as the crane during the first lifting movement; Get the The first lifting movement The surveillance image The crane arm pixel and the The specific calculation formula for the final distance of a transmission line is as follows: In the formula, Indicates The first lifting movement The surveillance image The crane arm pixel and the The final distance of the transmission line, Indicates The first lifting movement The surveillance image The crane arm pixel and the Transmission line No. Corrected distance of the crane's pixels on the same plane during the first lifting movement, Indicates The factor affecting the width and height distortion of the plane on which the crane is located during the first lifting movement.

10. A foreign body target perception system around a power transmission line, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of a method for sensing foreign objects around a power transmission line as described in any one of claims 1 to 9 are implemented.

Images