Power transmission line video image on-line monitoring method and system

Abstract

The invention discloses a power transmission line video image on-line monitoring method and system, and relates to the technical field of image analysis, and the method comprises the steps: analyzing the activity information of each activity region to determine the approaching range of the activity region, analyzing the activity conditions of different degrees in different activity regions, dividing the approaching range, and determining the approaching range. And a basis is provided for subsequent contact target object conditions of the power transmission line. The foreground area and the background area of the upper frame image of the monitoring video are divided according to the matching relation of the adjacent range, the monitoring video position and each activity area, and the timeliness and reliability of online monitoring of the power transmission line are improved according to the matching condition of the current related state characteristics of the power transmission line and the abnormal characteristic state template of the power transmission line. The precision of state monitoring of the power transmission line is ensured, and safety accidents of the power transmission line and normal operation of power are effectively avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and particularly to an online monitoring method and system for video images of transmission lines. Background Art

[0002] With the continuous expansion of the scale of the power system, transmission lines are widely distributed and the operating environment is complex. The traditional manual inspection method has low efficiency, high cost and is difficult to detect potential hazards in a timely manner. Under this background, the online monitoring technology of video images has emerged.

[0003] This technology integrates multiple fields of technology such as computer vision, image processing, sensors and communication. At the front end, high-definition cameras are used to collect images and videos of transmission lines and the surrounding environment in real time, and the states of key components such as conductors, insulators, and hardware fittings can be clearly captured. Through image processing algorithms, the image features can be automatically analyzed, such as detecting abnormal situations such as changes in conductor sag, insulator damage, and corrosion of hardware fittings.

[0004] At the same time, the collected data is transmitted to the monitoring center in real time through communication technology to achieve remote monitoring and early warning.

[0005] In the prior art, the monitoring of transmission lines is achieved through manual inspection, but there are time differences and errors in manual inspection, resulting in poor real-time performance and low accuracy of transmission line monitoring, and it cannot meet the requirements of online monitoring.

[0006] Therefore, how to improve the real-time performance and accuracy of transmission line monitoring is a technical problem to be solved at present. Summary of the Invention

[0007] The object of the present invention is to solve the problems of poor real-time performance and low accuracy in the monitoring of transmission lines in the prior art, and to propose an online monitoring method for video images of transmission lines, which includes, Collect the activity information and the monitoring video acquisition positions in the surrounding areas of the transmission line according to the position of the transmission line, divide the surrounding area of the transmission line into multiple activity areas according to the monitoring video acquisition positions, and establish the matching relationship between the monitoring video positions and each activity area; Analyze the activity information in each activity area to determine the adjacent range of the activity area, obtain the monitoring videos at each monitoring video position in real time, and divide the foreground area and the background area of the upper frame image of the monitoring video based on the adjacent range, the monitoring video position and the matching relationship between each activity area; Define and classify the abnormal states of the transmission line, establish an image sample set for each type of abnormal state of the transmission line, and extract the abnormal feature state template of the transmission line; Extract the current relevant state features of the transmission line from the foreground area and background area of the upper-frame image of the monitoring video, and realize the online monitoring of the transmission line according to the matching situation between the current relevant state features of the transmission line and the abnormal feature state template of the transmission line.

[0008] In some embodiments of the present application, divide the surrounding area of the transmission line into multiple activity areas according to the monitoring video acquisition location, and establish the matching relationship between the monitoring video acquisition location and each activity area, including, A plurality of monitoring video acquisition positions are evenly spaced according to the length of the transmission line, determine the monitoring range under each monitoring video acquisition position, and convert the monitoring range into actual area coordinates; According to the actual area coordinates under each monitoring video acquisition position, divide them into multiple first activity areas one by one, and record the area other than the first activity areas on the surrounding area of the transmission line as the second activity area, so as to establish the matching relationship between the monitoring video acquisition position and each first activity area and each second activity area; Among them, the activity area is the first activity area or the second activity area.

[0009] In some embodiments of the present application, analyze the activity information in each activity area to determine the proximity range of the activity area, including, Collect all activity event categories in the first activity area, screen out the activity event categories that affect the transmission line, and extract the activity characteristics and activity trajectories for each activity event category; Capture the activity feature change curve within a period of time under the activity event, split the activity feature change curve into multiple curve parts according to the change frequency on the activity feature change curve, calculate the slope change value of each curve part, count the maximum value, median value and minimum value of each curve part, and determine the representative value of each curve part based on the slope change value, maximum value, median value and minimum value of each curve part, so as to determine the representative value of each activity feature; Capture the activity trajectory within a period of time under the activity event, calculate the distance from each moment on the activity trajectory to the transmission line, and draw it into an activity trajectory distance curve, with the abscissa being time and the ordinate being the shortest distance between the activity trajectory and the transmission line at the corresponding time, and count the frequent distance intervals according to the activity trajectory distance curve, so as to determine the distance representative value of the activity trajectory; Combine the representative value of the activity feature and the distance representative value of the activity trajectory of the activity event type to determine the activity level of the first activity area, and thus map the proximity range of the first activity area.

[0010] In some embodiments of the present application, divide the foreground area and background area of the upper-frame image of the monitoring video by virtue of the proximity range, monitoring video position and the matching relationship with each activity area, including, Determine the frame rate of the monitoring video within the first activity area, construct an image frame sequence of the monitoring video, determine the time interval based on the activity level of the first activity area, and divide the image frame sequence of the monitoring video into multiple base frames according to the time interval; For each base frame, calculate the image similarity between adjacent base frames, determine the key frames according to the image similarity, and generate a new image frame sequence of the monitoring video based on the base frames and the key frames; For the new image frame sequence of the monitoring video, divide the foreground area and the background area of each frame of the image.

[0011] In some embodiments of the present application, for the new image frame sequence of the monitoring video, dividing the foreground area and the background area of each frame of the image includes, Perform Gaussian filtering on the frame images in the new image frame sequence of the monitoring video to smooth the images, calculate the gradient magnitude and direction of the image gray values, perform non-maximum suppression on the gradient magnitude to refine the edges, and obtain the edge contour of the transmission line body on the frame images through double-threshold processing and edge connection; Expand the corresponding distance outward with the edge contour of the transmission line body as the center according to the proximity range of the first activity area to obtain the adjacent area; Merge the edge contour of the transmission line body and its adjacent area to obtain the foreground area, and perform an image subtraction operation to subtract the foreground area from the entire frame image to obtain the background area.

[0012] In some embodiments of the present application, define and classify the abnormal states of the transmission line, establish an image sample set for each type of abnormal state of the transmission line, and extract the abnormal feature state templates of the transmission line, including, Divide the causes of the abnormal states of the transmission line into two types: internal and external, so as to divide the abnormal states of the transmission line into two types: abnormal caused by internal factors and abnormal caused by external factors; Collect the transmission line images corresponding to the abnormal states of the transmission line caused by internal and external factors; For the abnormal states caused by internal factors, use the edge contour of the transmission line body in the transmission line image as the image sample set in this abnormal state, and extract the image features on the image sample set to establish the abnormal feature state template for the abnormal states caused by internal factors; For the abnormal states caused by external factors, use the foreground area of the transmission line image as the image sample set in this abnormal state, and extract the image features on the image sample set to establish the abnormal feature state template for the abnormal states caused by external factors.

[0013] In some embodiments of the present application, extract the current relevant state features of the transmission line from the foreground area and the background area of the frame images in the monitoring video, including, Identify the foreground area and background area of the upper-frame image of the monitoring video, extract image features from the foreground area, and determine the dynamic features of the transmission line; Detect all target objects that have come into contact with the transmission line, extract the image features of the target objects from the background area, and determine the dynamic features of the target objects; The current relevant state features of the transmission line include the image features of the foreground area, the dynamic features of the transmission line, the target objects, the dynamic features of the target objects, and the image features of the target objects in the background area.

[0014] In some embodiments of the present application, according to the matching situation between the current relevant state features of the transmission line and the abnormal feature state template of the transmission line, including, Analyze the dynamic features of the target objects in the second activity area, combine the dynamic features in the first activity area and the second activity area to generate the overall dynamic features of the target objects; Calculate the matching degree with the abnormal feature state template of the transmission line according to the image features of the foreground area, the dynamic features of the transmission line, the target objects, the overall dynamic features of the target objects, and the image features of the target objects in the background area.

[0015] Correspondingly, the present application also provides an on-line monitoring system for transmission line video images, including, The first module is used to collect the activity information and the monitoring video acquisition position of the surrounding area of the transmission line according to the position of the transmission line, divide the surrounding area of the transmission line into multiple activity areas according to the monitoring video acquisition position, and establish the matching relationship between the monitoring video position and each activity area; The second module is used to analyze the activity information in each activity area to determine the proximity range of the activity area, obtain the monitoring video at each monitoring video position in real time, and divide the foreground area and the background area of the upper-frame image of the monitoring video by virtue of the proximity range, the monitoring video position and the matching relationship between each activity area; The third module is used to define and classify the abnormal states of the transmission line, establish an image sample set of the abnormal states of each type of transmission line, and extract the abnormal feature state template of the transmission line; The fourth module is used to extract the current relevant state features of the transmission line from the foreground area and the background area of the upper-frame image of the monitoring video, and realize the on-line monitoring of the transmission line according to the matching situation between the current relevant state features of the transmission line and the abnormal feature state template of the transmission line.

[0016] Compared with the prior art, the beneficial effects of the present invention are: 1. Divide the areas around the transmission line into multiple activity areas according to the collection positions of the monitoring videos, including a first activity area and a second activity area. The first activity area is the area directly monitored by the monitoring video, and the second activity area is the area that cannot be collected by the monitoring. The second activity area can assist in inferring the activity situation of the target object and facilitate the analysis of its contact with the transmission line. Analyze the activity information in each activity area to determine the proximity range of the activity area, analyze the different degrees of activity in different activity areas, divide the proximity range, and provide a basis for the subsequent situation of the target object in contact with the transmission line.

[0017] 2. Divide the foreground area and the background area of the frame image on the monitoring video based on the matching relationship between the proximity range, the monitoring video position, and each activity area. Define the foreground area and the background area on the frame image to facilitate the subsequent identification and monitoring of abnormal state types. According to the matching situation between the current relevant state characteristics of the transmission line and the abnormal characteristic state template of the transmission line, the timeliness and reliability of the online monitoring of the transmission line are improved, the accuracy of the state monitoring of the transmission line is ensured, and the occurrence of safety accidents of the transmission line and the normal operation of the power are effectively avoided. Description of the Drawings

[0018] Figure 1 It is a schematic flowchart of the online monitoring method for the video image of the transmission line proposed by the present invention; Figure 2 It is a schematic structural diagram of the online monitoring system for the video image of the transmission line proposed by the present invention. Detailed Embodiment

[0019] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0020] Refer to Figure 1 , the online monitoring method for the video image of the transmission line includes the following steps: Step S101, collect the activity information and the monitoring video collection position of the area around the transmission line according to the position of the transmission line, divide the area around the transmission line into multiple activity areas according to the monitoring video collection position, and establish a matching relationship between the monitoring video position and each activity area.

[0021] In this embodiment, the accurate position of the transmission line is obtained through a Geographic Information System (GIS). Use means such as sensor networks, drone inspections, and ground monitoring to collect the activity information of the area around the transmission line, including but not limited to personnel activities, vehicle movements, construction activities, etc. Record the positions of all monitoring videos to ensure that these positions can cover the transmission line and its surrounding key areas.

[0022] In some embodiments of the present application, the area around the transmission line is divided into multiple activity areas according to the monitoring video acquisition positions, and the matching relationship between the monitoring video acquisition positions and each activity area is established, including A plurality of monitoring video acquisition positions are evenly spaced according to the length of the transmission line, the monitoring range under each monitoring video acquisition position is determined, and the monitoring range is converted into actual area coordinates; According to the actual area coordinates under each monitoring video acquisition position, they are divided into multiple first activity areas one by one, and the area on the periphery of the transmission line except the first activity areas is denoted as the second activity area, so as to establish the matching relationship between the monitoring video acquisition position and each first activity area and each second activity area; Wherein, the activity area is the first activity area or the second activity area.

[0023] In this embodiment, because the transmission line is generally long, a plurality of monitoring video acquisition points are arranged at intervals. The acquisition blank area between the monitoring video acquisition points is the second activity area. The speculated activity conditions of the target objects (personnel, vehicles and other moving objects) in the blank area can be analyzed through the videos of adjacent areas. According to the length of the transmission line, a plurality of monitoring video acquisition positions are evenly spaced. Determine the monitoring range under each monitoring video acquisition position. Usually, according to parameters such as the viewing angle and focal length of the camera, the monitoring range is converted into actual area coordinates. According to the actual area coordinates under each monitoring video acquisition position, they are divided into multiple first activity areas one by one. The first activity area is the area directly monitored by the monitoring video. The area on the periphery of the transmission line except the first activity areas (monitoring interval area, i.e., blank area) is denoted as the second activity area. Establish the matching relationship between the monitoring video acquisition position and each first activity area and each second activity area, and form a corresponding data structure, such as a dictionary or a database table, for subsequent query and use.

[0024] It should be noted that subsequent analysis and processing of video images are all carried out for the first activity area (directly acquired by the monitoring video), and the content of the second activity area is the activity conditions of the target objects speculated and predicted based on the situations of adjacent first activity areas.

[0025] Step S102, analyze the activity information in each activity area to determine the adjacent range of the activity area, and obtain the monitoring video under each monitoring video position in real time. Divide the foreground area and the background area of the upper frame image of the monitoring video according to the adjacent range, the monitoring video position and the matching relationship between each activity area.

[0026] In this embodiment, in-depth analysis is performed on the activity information in each activity area, including activity type, frequency, intensity, etc. This adjacent range refers to a certain range around the transmission line, enabling the capture of the intersection situation between the target object and the transmission line. For example, it can accurately capture illegal situations such as damage and theft of the transmission line by some criminals. The monitoring video frame images at each monitoring video position are obtained in real time. Using image processing technology, combined with the matching relationship between the adjacent range, the monitoring video position, and the activity area, the foreground area and the background area are divided on the frame image. The foreground area includes the transmission line body part and the area within the adjacent range; the background area is the other part of the image except the foreground area.

[0027] In some embodiments of the present application, the activity information in each activity area is analyzed to determine the adjacent range of the activity area, including Collect all activity event categories in the first activity area, screen out the activity event categories that affect the transmission line, and extract the activity characteristics and activity trajectories for each activity event category; Capture the activity characteristic change curve within a period of time under the activity event, split the activity characteristic change curve into multiple curve parts according to the change frequency on the activity characteristic change curve, calculate the slope change value of each curve part, count the maximum value, median value, and minimum value of each curve part, and determine the representative value of each curve part based on the slope change value, maximum value, median value, and minimum value of each curve part, so as to determine the representative value of each activity characteristic; Capture the activity trajectory within a period of time under the activity event, calculate the distance from each moment on the activity trajectory to the transmission line, and draw it into an activity trajectory distance curve. The abscissa is time, and the ordinate is the shortest distance between the activity trajectory and the transmission line at the corresponding time. According to the activity trajectory distance curve, count the frequent distance intervals, so as to determine the distance representative value of the activity trajectory; Combine the representative value of the activity characteristic and the distance representative value of the activity trajectory of the activity event type to determine the activity level of the first activity area, and thus map out the adjacent range of the first activity area.

[0028] In this embodiment, all activity event categories in the first activity area are collected, such as personnel construction, vehicle driving, etc. Activity event categories that affect the transmission line are screened out. For example, personnel construction and vehicle driving may pose risks such as collisions to the transmission line. For each screened activity event category, activity features (such as the number of personnel, stay time, activity frequency, vehicle speed, etc.) and activity trajectories are extracted. There are two attributes of activity events that most affect the transmission line, one is the parameter of the activity feature, and the other is the distance between the activity trajectory and the transmission line. Based on the slope change value, maximum value, median value, and minimum value of each curve part, the representative value of each curve part (representative value of the curve) is determined, and thus the representative value of each activity feature (Representative value of activity features) is determined. The specific calculation formula is as follows: ; Wherein, is the representative value of the th activity feature, is the number of curve parts, , are respectively the two combination weights of the th curve part, , , are respectively the median value, maximum value, and minimum value of the th activity feature under the th curve part, is the slope change value of the th activity feature under the th curve part, is a preset constant, represents the correction of the representative value determined by the median value, maximum value, and minimum value due to the slope change situation, is the magnitude of the balance correction function. An initial representative value is determined according to the median value, maximum value, and minimum value. According to the correction of the initial representative value due to the slope change situation, the more stable the change, the lower the correction degree. Because of the uncertainty and complexity of the activity time, this representative value represents the degree of this activity feature in the average case.

[0029] In this embodiment, the distance from each moment on the activity trajectory to the transmission line is calculated. Here, the distance is the shortest distance from the activity trajectory point to the transmission line at any moment. The frequent distance intervals (the intervals with the most occurrences) are statistically obtained to determine the distance representative value of the activity trajectory. The activity level of the first activity area is determined by combining the representative value of the activity characteristics of the activity event type and the distance representative value of the activity trajectory. The specific calculation formula is as follows: ; Wherein, is the activity level of the th first activity area, , are the conversion coefficients of the activity characteristics and the activity trajectory respectively, , are the number of activity event types and the number of activity trajectories respectively, , are the weights of the rd activity event type and the th activity trajectory respectively, is the th comprehensive representative value of all activity characteristics of the st activity event type in the th first activity area, is the distance representative value of the th activity trajectory in the th first activity area, is a preset constant to balance the size of the activity level. The proximity range of the first activity area is mapped through the activity level. Generally speaking, the higher the activity level, the larger the proximity range, and [] is the rounding symbol.

[0030] In some embodiments of the present application, the foreground area and the background area of the upper frame image of the surveillance video are divided by virtue of the matching relationship between the proximity range, the surveillance video position and each activity area, including, Determine the frame rate of the surveillance video within each first activity area, construct an image frame sequence of the surveillance video, determine the time interval according to the activity level of the first activity area, and divide the image frame sequence of the surveillance video into multiple basic frames according to the time interval; For each basic frame, calculate the image similarity between adjacent basic frames, determine the key frames according to the image similarity, and generate a new image frame sequence of the surveillance video according to the basic frames and the key frames; For the new image frame sequence of the surveillance video, divide the foreground area and the background area of each frame image.

[0031] In this embodiment, since there is a large amount of repetitive content in the images on the surveillance video, in order to balance the real-time processing speed and the accuracy of image analysis, it is necessary to screen the images. The time interval is determined by the activity level of the first activity area. Different activity levels correspond to different time intervals. For each base frame, the image similarity between adjacent base frames is calculated. Methods such as mean square error (MSE) or structural similarity (SSIM) can be used. Key frames are determined according to the image similarity. For example, a similarity threshold is set. When the similarity between adjacent frames is lower than this threshold, the current frame is determined as a key frame. An image frame sequence of a new surveillance video is generated based on the base frames and the key frames.

[0032] In some embodiments of the present application, for the image frame sequence of the new surveillance video, the foreground area and the background area of each frame image are divided, including, Perform Gaussian filtering on the frame images in the image frame sequence of the new surveillance video to smooth the images, calculate the gradient magnitude and direction of the image gray value, perform non-maximum suppression on the gradient magnitude to refine the edges, and obtain the edge contour of the transmission line body on the frame image through double threshold processing and edge connection; According to the proximity range of the first activity area, expand the corresponding distance outward with the edge contour of the transmission line body as the center to obtain a proximity area; Merge the edge contour of the transmission line body and its proximity area to obtain the foreground area, and through an image subtraction operation, subtract the foreground area from the entire frame image to obtain the background area.

[0033] In this embodiment, this proximity area may be the area where the target object intersects or contacts the transmission line.

[0034] Step S103, define and classify the abnormal states of the transmission line, establish an image sample set for each type of abnormal state of the transmission line, and extract the abnormal feature state template of the transmission line.

[0035] In some embodiments of the present application, define and classify the abnormal states of the transmission line, establish an image sample set for each type of abnormal state of the transmission line, and extract the abnormal feature state template of the transmission line, including, The reasons for the abnormal states of the transmission line are divided into two types: internal and external, so the abnormal states of the transmission line are divided into two types: abnormal caused by internal reasons and abnormal caused by external reasons; Collect the transmission line images corresponding to the abnormal states of the transmission line caused by internal and external reasons; For the abnormal states caused by internal reasons, use the edge contour of the transmission line body in the transmission line image as the image sample set in this abnormal state, and extract the image features on the image sample set to establish the abnormal feature state template for the abnormal states caused by internal reasons; For externally caused anomalies, the foreground area of the transmission line image is used as the image sample set in this abnormal state, and the image features on the image sample set are extracted to establish an abnormal feature state template for the externally caused abnormal state.

[0036] In this embodiment, the transmission line includes poles, conductors, insulators, etc. Internally caused anomalies, such as those brought about by aging, environmental humidity, etc., and externally caused anomalies, such as man-made damage, damage by other moving objects, etc. For internally caused anomalies, the edge contour of the transmission line body in the transmission line image is used as the image sample set in this abnormal state, and the image features on the image sample set (such as the curvature and length of the edge) are extracted to establish an abnormal feature state template for the internally caused abnormal state. For externally caused anomalies, the foreground area of the transmission line image is used as the image sample set in this abnormal state, and the image features on the image sample set (such as the position and size of the target object) are extracted to establish an abnormal feature state template for the externally caused abnormal state.

[0037] It can be understood that for anomalies caused by internal reasons, only the image features within the edge contour area of the transmission line body need to be counted. For anomalies caused by external reasons, it is necessary to analyze the intersection situation between the target object (contact object) and the transmission line, and additional image features of the adjacent area need to be added, that is, analyze by integrating the image features of the foreground area.

[0038] Step S104, extract the current relevant state features of the transmission line from the foreground area and background area of the previous frame image of the monitoring video, and realize the online monitoring of the transmission line according to the matching situation between the current relevant state features of the transmission line and the abnormal feature state template of the transmission line.

[0039] In this embodiment, the current relevant state features not only include the image features of the transmission line, but also the dynamic features brought about by the video time continuity (such as the dynamic changes in the line position, line swing, which may be caused by external factors such as wind), the image features and dynamic features of the target object, that is, the dynamic changes of the moving object in contact with the transmission line (such as the vehicle direction and speed, etc.).

[0040] In some embodiments of the present application, extract the current relevant state features of the transmission line from the foreground area and background area of the previous frame image of the monitoring video, including, Identify the foreground area and background area of the previous frame image of the monitoring video, extract image features on the foreground area, and determine the dynamic features of the transmission line; Detect all target objects that have come into contact with the transmission line, extract the image features of the target objects on the background area, and determine the dynamic features of the target objects; The current relevant state features of the transmission line include the image features of the foreground area, the dynamic features of the transmission line, the target objects, the dynamic features of the target objects, and the image features of the target objects in the background area.

[0041] In this embodiment, the foreground area and the background area of the upper-frame image of the monitoring video are identified. Image features (such as the morphological changes of the transmission line and the damage condition of the insulators) are extracted from the foreground area, and the dynamic features of the transmission line (such as the vibration frequency and swing amplitude of the wire) are determined. All target objects that have come into contact with the transmission line are detected. Image features of the target objects (such as the shape and color of the target objects) are extracted from the background area, and the dynamic features of the target objects (such as the moving speed and direction of the target objects) are determined. The current relevant state features of the transmission line include the image features of the foreground area, the dynamic features of the transmission line, the target objects, the dynamic features of the target objects, and the image features of the target objects in the background area.

[0042] In some embodiments of the present application, according to the matching situation between the current relevant state features of the transmission line and the abnormal feature state template of the transmission line, including, Analyze the dynamic features of the target objects in the second activity area, and combine the dynamic features in the first activity area and the second activity area to generate the overall dynamic features of the target objects; Calculate the matching degree with the abnormal feature state template of the transmission line according to the image features of the foreground area, the dynamic features of the transmission line, the target objects, the overall dynamic features of the target objects, and the image features of the target objects in the background area.

[0043] In this embodiment, the image features of the foreground area are mainly used to analyze the current state of the transmission line, judge whether the current state is abnormal and whether there are contacting target objects. If there are contacting target objects, analyze the trajectory of the target objects from the inside out. First, determine the trajectory of the target objects in the foreground area, and then determine the trajectory of the target objects in the background area to obtain the activity situation of the target objects in the first activity area, and then infer the activity situation of the target objects in the adjacent second activity area. If there are no target objects in the foreground area that contact the transmission line, the reasons for abnormalities caused by external factors are not considered.

[0044] Use object detection algorithms (such as YOLO, Faster R-CNN, etc.) to detect target objects in the image of the first activity area and determine their positions (usually represented by bounding boxes). These algorithms can accurately identify target objects in complex backgrounds. Extract the contour information of the target objects, which can be achieved through edge detection algorithms (such as Canny edge detection). The contour information helps to further analyze the shape and pose of the target objects.

[0045] Motion trajectory analysis: If there are multiple frames of images of the first active area, the motion trajectory of the target object can be analyzed by tracking its position changes in different frames. For example, motion parameters such as the speed and acceleration of the target object can be calculated.

[0046] Shape change analysis: Compare the shapes of the target object at different times (which can be represented by contours or shape feature descriptors), and analyze whether its shape has changed and the degree of change.

[0047] Posture change analysis: For target objects with obvious postures (such as people, animals, etc.), the relative positional relationship of their body parts can be analyzed to determine whether their postures have changed.

[0048] Analyze the spatial positional relationship between the first active area and the second active area. Generally, the two adjacent sides of the second active area are the first active area. For example, if the two active areas are adjacent and have a certain spatial continuity, the possible position of the target object in the second active area can be inferred based on the motion direction and speed of the target object in the first active area.

[0049] Consider the influence of environmental factors on the motion of the target object, such as terrain, obstacles, etc. If there are terrain changes or obstacles between the first active area and the second active area, the inference of the motion of the target object in the second active area needs to be adjusted according to these factors.

[0050] In this embodiment, for the anomalies caused by internal reasons, only the matching degree between the image features of the transmission line body part and the dynamic features of the transmission line is considered. First, the basic matching degree is obtained according to the matching situation between the image features of the transmission line body part and the template, and the final matching degree is obtained by adjusting the basic matching degree through the dynamic features (different adjustment coefficients correspond to different degrees of comprehensive dynamic features, and the adjustment is completed by adjustment coefficient * basic matching degree). For the anomalies caused by external reasons, the matching degree is calculated by considering the image features of the foreground area, the dynamic features of the transmission line, the target object, the overall dynamic features of the target object, and the image features of the target object in the background area. The contact risk of the target object is analyzed according to the target object, the overall dynamic features of the target object, and the image features of the target object in the background area. The basic matching degree is obtained according to the matching situation between the image features of the foreground area and the template, and the final matching degree is determined according to the contact risk of the target object, the dynamic features of the transmission line, and the basic matching degree. The specific calculation formula is as follows: ; Wherein, is the final matching degree, is the basic matching degree, is the number of dynamic features, is the th influence weight of the dynamic feature, is the size of the th dynamic feature, is the contact risk of the target object, is a preset constant, represents the correction of the sum of the matching degrees of the contact risk of the target object to the image features and the dynamic features; It should be noted that the dynamic features of the target object extracted from the first activity area and the change situation of the target object in the second activity area inferred are integrated. Specifically, it includes motion trajectory, speed, acceleration, shape change, posture change, etc. These features reflect the motion state and trend of the target object. Risk identification, spatial proximity: Analyze the spatial position relationship between the motion trajectory of the target object and the transmission line, and judge whether the target object is likely to approach or contact the transmission line. Speed and acceleration: Consider the speed and acceleration of the target object, and evaluate its approaching rate and potential impact force on the transmission line. Shape and posture: Analyze the shape and posture of the target object, and judge whether it is easy to contact the transmission line (for example, long strip objects are more likely to hook the wire). Based on the overall dynamic features of the target object and the image features of the target object in the background area, calculate the probability of the target object contacting the transmission line. This can be achieved by establishing a probability model, and the model can consider the comprehensive effect of various factors to determine the risk.

[0051] Correspondingly, the present application also provides an on-line monitoring system for transmission line video images, as Figure 2 shown, including, The first module is used to collect the activity information of the area around the transmission line and the monitoring video acquisition position according to the position of the transmission line, divide the area around the transmission line into multiple activity areas according to the monitoring video acquisition position, and establish the matching relationship between the monitoring video position and each activity area; The second module is used to analyze the activity information in each activity area to determine the proximity range of the activity area, obtain the monitoring video at each monitoring video position in real time, and divide the foreground area and the background area of the upper frame image of the monitoring video according to the proximity range, the monitoring video position and the matching relationship between each activity area; The third module is used to define and classify the abnormal states of the transmission line, establish an image sample set of the abnormal states of each type of transmission line, and extract the template of the abnormal feature state of the transmission line; The fourth module is used to extract the current relevant state features of the transmission line in the foreground area and the background area of the upper frame image of the monitoring video, and realize the on-line monitoring of the transmission line according to the matching situation between the current relevant state features of the transmission line and the template of the abnormal feature state of the transmission line.

[0052] Compared with the prior art, the beneficial effects of the present invention are: 1. Divide the area around the transmission line into multiple activity areas according to the location of the monitored video collection, including the first activity area and the second activity area. The first activity area is the area directly monitored by the monitored video, and the second activity area is the area that cannot be monitored. The second activity area can assist in inferring the activity of the target object and facilitate the analysis of its contact with the transmission line. Analyze the activity information in each activity area to determine the adjacent range of the activity area, analyze the different degrees of activity in different activity areas, and divide the adjacent range to provide a basis for the subsequent situation of the target object in contact with the transmission line.

[0053] 2. Divide the foreground area and the background area of the upper frame image of the monitored video based on the matching relationship between the adjacent range, the position of the monitored video, and each activity area. Define the foreground area and the background area on the frame image to facilitate the subsequent identification and monitoring of abnormal state types. According to the matching situation between the current relevant state characteristics of the transmission line and the abnormal characteristic state template of the transmission line, the timeliness and reliability of the online monitoring of the transmission line are improved, the accuracy of the transmission line state monitoring is ensured, and the occurrence of safety accidents of the transmission line and the normal operation of the power are effectively avoided.

[0054] Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by hardware or by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various implementation scenarios of the present invention.

[0055] Those skilled in the art can understand that the drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the drawings are not necessarily essential for implementing the present invention.

[0056] Those skilled in the art can understand that the modules in the system in the implementation scenario can be distributed in the system in the implementation scenario according to the description of the implementation scenario, or can be correspondingly changed and located in one or more systems different from the present implementation scenario. The modules in the above implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0057] The above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution of the present invention and its inventive concept, makes an equivalent substitution or change, and should be covered by the protection scope of the present invention.

Claims

1. An on-line monitoring method for video images of transmission lines, characterized in that, including collecting the activity information and the monitoring video acquisition locations in the areas around the transmission lines according to the positions of the transmission lines, dividing the areas around the transmission lines into multiple activity areas according to the monitoring video acquisition locations, and establishing the matching relationship between the monitoring video locations and each activity area; analyzing the activity information in each activity area to determine the proximity range of the activity area, obtaining the monitoring videos at each monitoring video location in real time, and dividing the foreground area and the background area of the upper frame image of the monitoring video based on the proximity range, the monitoring video location, and the matching relationship between each activity area; defining and classifying the abnormal states of the transmission lines, establishing the image sample sets of the abnormal states of each type of transmission lines, and extracting the template of the abnormal characteristic states of the transmission lines; extracting the current relevant state characteristics of the transmission lines in the foreground area and the background area of the upper frame image of the monitoring video, and realizing the online monitoring of the transmission lines according to the matching situation between the current relevant state characteristics of the transmission lines and the template of the abnormal characteristic states of the transmission lines.

2. The on-line monitoring method for video images of transmission lines according to claim 1, characterized in that, Dividing the areas around the transmission lines into multiple activity areas according to the monitoring video acquisition locations, and establishing the matching relationship between the monitoring video acquisition locations and each activity area including setting a plurality of monitoring video acquisition locations at uniform intervals according to the length of the transmission line, determining the monitoring range at each monitoring video acquisition location, and converting the monitoring range into actual area coordinates; dividing each actual area coordinate at each monitoring video acquisition location into multiple first activity areas one by one, and denoting the area other than the first activity areas in the areas around the transmission lines as the second activity areas, thereby establishing the matching relationship between the monitoring video acquisition locations and each first activity area and each second activity area; wherein, the activity area is the first activity area or the second activity area.

3. The on-line monitoring method for video images of transmission lines according to claim 2, characterized in that, Analyzing the activity information in each activity area to determine the proximity range of the activity area including collecting all the activity event categories in the first activity area, screening out the activity event categories that have an impact on the transmission lines, and extracting the activity characteristics and activity trajectories for each activity event category; capturing the change curve of the activity characteristics within a period of time under the activity event, splitting the change curve of the activity characteristics into multiple curve parts according to the change frequency on the change curve of the activity characteristics, calculating the slope change value of each curve part, counting the maximum value, median value, and minimum value of each curve part, and determining the representative value of each curve part based on the slope change value, maximum value, median value, and minimum value of each curve part, so as to determine the representative value of each activity characteristic; capturing the activity trajectory within a period of time under the activity event, calculating the distance from each moment on the activity trajectory to the transmission line, and plotting it into an activity trajectory distance curve, with the abscissa being time and the ordinate being the shortest distance from the activity trajectory to the transmission line at the corresponding time, and counting the frequent distance intervals based on the activity trajectory distance curve, so as to determine the distance representative value of the activity trajectory; combining the representative value of the activity characteristics and the distance representative value of the activity trajectory of the activity event type to determine the activity level of the first activity area, and mapping the proximity range of the first activity area accordingly.

4. The on-line monitoring method for video images of transmission lines according to claim 3, characterized in that Divide the foreground area and the background area of the upper-frame image of the monitoring video based on the matching relationship between the proximity range, the position of the monitoring video, and each activity area, including Determine the frame rate of the monitoring video within a first activity area, construct an image frame sequence of the monitoring video, determine the time interval based on the activity level of the first activity area, and divide the image frame sequence of the monitoring video into multiple basic frames according to the time interval; For each basic frame, calculate the image similarity between adjacent basic frames, determine the key frames according to the image similarity, and generate a new image frame sequence of the monitoring video based on the basic frames and the key frames; For the new image frame sequence of the monitoring video, divide the foreground area and the background area of each frame image.

5. The on-line monitoring method for video images of transmission lines according to claim 4, characterized in that, For the new image frame sequence of the monitoring video, divide the foreground area and the background area of each frame image, including Perform Gaussian filtering on the frame images in the new image frame sequence of the monitoring video to smooth the images, calculate the gradient magnitude and direction of the image gray value, perform non-maximum suppression on the gradient magnitude to refine the edges, and obtain the edge contour of the transmission line body on the frame image through double-threshold processing and edge connection; Expand the corresponding distance outward with the edge contour of the transmission line body as the center according to the proximity range of the first activity area to obtain the proximity area; Merge the edge contour of the transmission line body and its proximity area to obtain the foreground area, and perform an image subtraction operation to subtract the foreground area from the entire frame image to obtain the background area.

6. The on-line monitoring method for video images of transmission lines according to claim 5, characterized in that Define and classify the abnormal states of the transmission line, establish an image sample set for each type of abnormal state of the transmission line, and extract the abnormal feature state template of the transmission line, including Divide the causes of the abnormal states of the transmission line into two types: internal and external, so as to divide the abnormal states of the transmission line into two types: abnormal caused by internal factors and abnormal caused by external factors; Collect the transmission line images corresponding to the abnormal states of the transmission line caused by internal factors and external factors; For the abnormal states caused by internal factors, use the edge contour of the transmission line body in the transmission line image as the image sample set in this abnormal state, extract the image features on the image sample set, and establish the abnormal feature state template for the abnormal states caused by internal factors; For the abnormal states caused by external factors, use the foreground area of the transmission line image as the image sample set in this abnormal state, extract the image features on the image sample set, and establish the abnormal feature state template for the abnormal states caused by external factors.

7. The on-line monitoring method for video images of a transmission line according to claim 6, characterized in that Extract the current relevant state features of the transmission line from the foreground area and the background area of the upper-frame image of the monitoring video, including Identify the foreground area and the background area of the upper-frame image of the monitoring video, extract the image features on the foreground area, and determine the dynamic features of the transmission line; Detect all target objects that have come into contact with the transmission line, extract the image features of the target objects on the background area, and determine the dynamic features of the target objects; The current relevant state features of the transmission line include the image features of the foreground area, the dynamic features of the transmission line, the target objects, the dynamic features of the target objects, and the image features of the target objects in the background area.

8. The on-line monitoring method for video images of transmission lines according to claim 7, characterized in that, According to the matching situation between the current relevant state features of the transmission line and the abnormal feature state template of the transmission line, including Analyze the dynamic characteristics of the target object in the second activity area, combine the dynamic characteristics in the first and second activity areas to generate the overall dynamic characteristics of the target object; Calculate the matching degree with the abnormal feature state template of the transmission line according to the image features of the foreground area, the dynamic characteristics of the transmission line, the target object, the overall dynamic characteristics of the target object, and the image features of the target object in the background area.

9. Online monitoring system for video images of transmission lines, characterized in that, Including, The first module is used to collect the activity information of the surrounding area of the transmission line and the monitoring video acquisition location according to the position of the transmission line, divide the surrounding area of the transmission line into multiple activity areas according to the monitoring video acquisition location, and establish the matching relationship between the monitoring video location and each activity area; The second module is used to analyze the activity information in each activity area to determine the adjacent range of the activity area, obtain the monitoring video at each monitoring video location in real time, and divide the foreground area and the background area of the upper frame image of the monitoring video according to the adjacent range, the monitoring video location, and the matching relationship with each activity area; The third module is used to define and classify the abnormal states of the transmission line, establish an image sample set for each type of abnormal state of the transmission line, and extract the abnormal feature state template of the transmission line; The fourth module is used to extract the current relevant state features of the transmission line in the foreground area and the background area of the upper frame image of the monitoring video, and realize the online monitoring of the transmission line according to the matching situation between the current relevant state features of the transmission line and the abnormal feature state template of the transmission line.

Images