Flying and hanging object identification method and system for power transmission line

Abstract

The invention provides a method and a system for identifying a hanging object of a power transmission line, and the method comprises the steps: obtaining a real-time video stream, extracting a moving target from the real-time video stream, and obtaining a preliminary screening result; wherein the preliminary screening result at least comprises a related picture of the moving target; inputting the preliminary screening result into a preset training identification model to judge whether the moving target in the preliminary screening result is a power transmission line floating hanging object or not; wherein the identification model is trained through pre-stored floating and hanging object sample data and is used for identifying whether the moving target is a floating and hanging object. According to the invention, on the basis of inter-frame difference fusion deep learning, through real-time video analysis, the hanging object on the power transmission line can be rapidly and accurately identified. Power grid safety operation and maintenance level is improved, and power supply stability is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of identification of hanging objects on power lines, and in particular to a method and system for identification of hanging objects on power lines. Background Art

[0002] Transmission lines exposed to the complex and ever-changing natural environment are constantly facing various external interferences from natural factors such as wind, rain, and snow, as well as human activities. Among them, the problem of floating objects on transmission lines is particularly prominent. Once floating objects, such as plastic bags, kite lines, and construction waste, are attached to transmission lines, they may not only cause serious accidents such as line short circuits and tripping, but may also pose a long-term threat to the safety of the power grid, affecting the reliability and stability of power supply.

[0003] Given the significant impact of power transmission hanging objects on power grid safety, timely and accurate identification and elimination of these hidden dangers have become key issues that need to be urgently addressed in the field of power transmission operation and maintenance. At present, a series of studies have been conducted in academia and industry on the identification of power transmission hanging objects, and a variety of recognition algorithms have been proposed. However, most of these algorithms have problems with insufficient real-time performance or low recognition efficiency, making it difficult to achieve efficient and accurate detection of hanging objects in complex and changing actual environments. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for identifying hanging objects on a transmission line, so as to solve the technical problem of how to quickly and accurately identify hanging objects on a transmission line.

[0005] On the one hand, a method for identifying hanging objects on a transmission line is provided, comprising:

[0006] Acquire a real-time video stream, and extract a moving target from the real-time video stream to obtain a preliminary screening result; wherein the preliminary screening result at least includes a relevant picture of the moving target;

[0007] The preliminary screening results are input into a preset trained recognition model to determine whether the moving target in the preliminary screening results is a hanging object on the power transmission line; wherein the recognition model is trained with pre-stored hanging object sample data to identify whether the moving target is a hanging object.

[0008] Preferably, extracting the moving target from the real-time video stream specifically includes:

[0009] Obtain three consecutive frames of images from a real-time video stream, calculate the inter-frame differences of the three consecutive frames of images, and determine the corresponding images after the frame differences;

[0010] Performing binarization and addition operation on the image after frame difference, and performing grayscale projection on the image after operation to obtain the corresponding grayscale projection curve;

[0011] If the grayscale projection peak value at the target position is greater than the grayscale projection value of the noise point, it is determined that there is a moving target at the target position.

[0012] Preferably, the inter-frame difference of three consecutive frames of images is calculated according to the following formula:

[0013] G1(x,y)=|F k (x,y)-F k-1 (x,y)|

[0014] G2(x,y)=|F k+1 (x,y)-F k (x,y)|

[0015] Among them, F k-1 (x,y), F k (x,y), F k+1 (x, y) is the k-1th, kth, k+1th consecutive frame image; G1(x, y), G2(x, y) are the images after frame difference.

[0016] Preferably, the image after frame difference is binarized and added according to the following formula:

[0017] R(x,y)=G1(x,y)+G2(x,y)

[0018] Among them, R(x,y) is the image after the addition operation.

[0019] Preferably, the preset trained recognition model at least includes:

[0020] A feature extraction model is used to extract feature information of an input image according to preset rules;

[0021] Feature fusion model, used to fuse the extracted feature information to obtain feature values;

[0022] A detection head model, used to set a corresponding detection head according to the characteristic value;

[0023] The suppression model is used to extract the prediction boxes and scores of each category whose scores are greater than a preset threshold according to the set detection head, and perform non-maximum suppression using the positions and scores of the prediction boxes.

[0024] Preferably, the preset trained recognition model is specifically used to divide the input related picture of the moving target into a number of pixels;

[0025] Place an anchor point on each pixel, set multiple detection heads based on the anchor points, and set upper and lower boundaries for the detection heads;

[0026] The anchor point is responsible for predicting the target position and ultimately outputting the predicted frame that best matches the true frame.

[0027] Preferably, pre-training is performed by the following steps:

[0028] Establish a sample database of floating objects;

[0029] Construct the corresponding network model and train it based on the pre-stored data set to obtain a converged model as a pre-training model;

[0030] Load the pre-trained model, perform transfer learning training based on the sample database, and use the floating object evaluation criteria to evaluate the test data of the pre-trained model in the test set;

[0031] The model corresponding to the optimal test value is saved to obtain the optimal model.

[0032] On the other hand, a system for identifying hanging objects on a power line is also provided, which is used to implement the method for identifying hanging objects on a power line, comprising:

[0033] An image processing module is used to obtain a real-time video stream and extract a moving target from the real-time video stream to obtain a preliminary screening result; wherein the preliminary screening result at least includes relevant images of the moving target;

[0034] The recognition module is used to input the preliminary screening results into a preset trained recognition model to determine whether the moving target in the preliminary screening results is a hanging object on the transmission line; wherein the recognition model is trained with pre-stored hanging object sample data to identify whether the moving target is a hanging object.

[0035] Preferably, the image processing module is specifically used to obtain three consecutive frames of images from the real-time video stream, and calculate the inter-frame difference of the three consecutive frames of images to determine the corresponding image after the frame difference;

[0036] Performing binarization and addition operation on the image after frame difference, and performing grayscale projection on the image after operation to obtain the corresponding grayscale projection curve;

[0037] If the grayscale projection peak value at the target position is greater than the grayscale projection value of the noise point, it is determined that there is a moving target at the target position;

[0038] The inter-frame difference of three consecutive frames of images is calculated according to the following formula:

[0039] G1(x,y)=|F k (x,y)-F k-1 (x,y)|

[0040] G2(x,y)=|F k+1 (x,y)-F k (x,y)|

[0041] Among them, F k-1(x,y), F k (x,y), F k+1 (x, y) is the k-1th, kth, k+1th consecutive frame image; G1(x, y), G2(x, y) are the images after frame difference;

[0042] The image after frame difference is binarized and added according to the following formula:

[0043] R(x,y)=G1(x,y)+G2(x,y)

[0044] Among them, R(x,y) is the image after the addition operation.

[0045] Preferably, the recognition module is specifically used to divide the input related picture of the moving target into a number of pixels;

[0046] Place an anchor point on each pixel, set multiple detection heads based on the anchor points, and set upper and lower boundaries for the detection heads;

[0047] The anchor point is responsible for predicting the target position and ultimately outputting the predicted frame that best matches the true frame.

[0048] In summary, the implementation of the embodiments of the present invention has the following beneficial effects:

[0049] The method and system for identifying hanging objects on transmission lines provided by the present invention can quickly and accurately identify hanging objects on transmission lines through real-time video analysis based on inter-frame difference fusion deep learning, thereby improving the safe operation and maintenance level of power grids and ensuring stable power supply. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, without paying creative labor, other drawings obtained based on these drawings still belong to the scope of the present invention.

[0051] Figure 1 The main flow diagram of a method for identifying hanging objects on a transmission line according to an embodiment of the present invention is shown in FIG.

[0052] Figure 2 The figure is a logical schematic diagram of a method for identifying hanging objects on a transmission line according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings.

[0054] like Figure 1and Figure 2 FIG. 1 is a schematic diagram of an embodiment of a method for identifying hanging objects on a transmission line provided by the present invention. In this embodiment, the method comprises the following steps:

[0055] Step S1, obtain the real-time video stream, and extract the moving target from the real-time video stream to obtain the preliminary screening result; wherein, the preliminary screening result at least includes the relevant pictures of the moving target; understandably, the inter-frame difference operation is performed on three consecutive frames, the logical "and" operation in the traditional three-frame difference method is abandoned, and the frame difference image is synthesized by addition operation. The characteristic performance of the moving target is effectively enhanced, and the information loss caused by the logical operation is reduced. Then, the grayscale projection processing is performed on the frame difference image after the addition operation. By calculating the cross-correlation value between adjacent frame projections, the position change of the moving target in the continuous frame can be accurately locked, thereby realizing the accurate detection and recognition of the moving target; not only inherits the high real-time performance of the inter-frame difference method, but also significantly improves the recognition accuracy through the fine analysis of the grayscale projection, providing strong technical support for the rapid response to dynamic hidden dangers such as power transmission hanging objects.

[0056] In one embodiment, the extraction of moving targets from the real-time video stream specifically includes obtaining three consecutive frames of images from the real-time video stream, calculating the inter-frame difference of the three consecutive frames of images, and determining the corresponding image after the frame difference; performing a binarization addition operation on the image after the frame difference, and performing a grayscale projection on the image after the operation to obtain a corresponding grayscale projection curve; if the grayscale projection peak value at the target position is greater than the grayscale projection value of the noise point, it is determined that there is a moving target at the target position. It can be understood that the improved three-frame difference algorithm is used to enhance the feature performance of the moving target, and the displacement of the moving target is accurately located through grayscale projection processing and the projection relationship between adjacent frames; at the same time, the PPYOLOE_PLUS algorithm is used to extract features based on the CSPRepStage structure, and the PANET network is used for feature fusion, and the attention mechanism is added through the ESE structure to enhance the detection head's attention to effective features. Finally, through dynamic positive and negative sample allocation, loss function calculation and non-maximum suppression, the predicted frame with the highest fit with the real frame is output, and efficient detection and recognition of moving targets are achieved.

[0057] Specifically, the inter-frame difference of three consecutive frames of images is calculated according to the following formula:

[0058] G1(x,y)=|F k (x,y)-F k-1 (x,y)|

[0059] G2(x,y)=|F k+1 (x,y)-F k (x,y)|

[0060] Among them, F k-1 (x,y), F + (x,y), F ++1 (x, y) is the k-1th, kth, k+1th consecutive frame image; G1(x, y), G2(x, y) are the images after frame difference.

[0061] The image after frame difference is binarized and added according to the following formula:

[0062] R(x,y)=G1(x,y)+G2(x,y)

[0063] Among them, R(x,y) is the image after the addition operation.

[0064] In one embodiment, the grayscale projection algorithm is a motion estimation algorithm that estimates the motion of an image based on the grayscale curve of the image. It makes full use of the change law of the overall grayscale distribution of the image to determine the motion vector, and only needs to perform a correlation operation on the projection curve of the image rows and columns to calculate the motion vector, thereby reducing the amount of calculation and ensuring a high calculation speed. Take the kth image in the sequence of images as the reference frame, and perform cumulative projection of the grayscale values ​​of the image rows and columns:

[0065]

[0066] Where: f k (i, j) is the gray value of point (i, j) in the reference frame image; C k (j) is the cumulative projection of the grayscale value of the jth column of pixels in the reference frame image; R k (i) is the cumulative projection of the grayscale values ​​of the pixels in the i-th row in the reference frame image.

[0067] After the addition operation, although the noise points in the adjacent frame images increase, the grayscale value of the moving target is much larger than the noise points. In the grayscale projection curve, the grayscale projection peak value at the moving target position will definitely be larger than the grayscale projection value of the noise points. The present invention adopts the inter-frame difference and grayscale projection fusion algorithm, performs grayscale projection on the frame difference addition image, obtains the horizontal and vertical projection curves, and uses the cross-correlation operation to determine the peak coordinates, that is, the displacement of the moving target between adjacent frames, thereby realizing the detection of the moving target.

[0068] Step S2, input the preliminary screening results into a preset trained recognition model to determine whether the moving target in the preliminary screening results is a floating object on the transmission line; wherein the recognition model is trained by pre-stored floating object sample data to identify whether the moving target is a floating object. It can be understood that common hanging objects (kites, plastic bags, etc.) are taken as independent detection targets, and the detection of hanging objects is achieved through a deep learning target detection algorithm.

[0069] In one embodiment, the preset trained recognition model at least includes a feature extraction model for extracting feature information of the input image according to preset rules; a feature fusion model for fusing the extracted feature information to obtain feature values; a detection head model for setting the corresponding detection head according to the feature value; and a suppression model for extracting the prediction box and score of each category with a score greater than a preset threshold according to the set detection head, and performing non-maximum suppression using the position and score of the prediction box. The overall network is divided into four parts. The first part, Backbone, is the feature extraction part, including the CSPRepReStage structure and the ESE structure; wherein CSPRepReStage combines the characteristics of the CSP and RepVgg structures, and reduces the information loss and calculation amount in the feature transmission process while further improving the feature extraction capability; the second part, Neck, is the feature fusion part, which is composed of PANET. The PANET network adds an additional information transmission path on the basis of FPN, enriching the feature learning range of the network; the third part, Head, is the detection head part, which mainly includes the ESE structure. The ESE structure adds an attention mechanism on the basis of CSPRepReStage, which can better focus on effective features. The fourth part, Prediction, includes dynamic positive and negative sample allocation, loss function calculation and non-maximum suppression. Take out the boxes and scores of each category whose scores are greater than the threshold threshold, and use the box positions and scores for non-maximum suppression.

[0070] In one embodiment, the preset trained recognition model is specifically used to divide the input moving target related picture into several pixels; place an anchor point on each pixel, and set multiple detection heads according to the anchor point, and set upper and lower boundaries for the detection heads; the anchor point is responsible for predicting the target position, and finally outputs the predicted frame with the highest degree of fit with the real frame. It can be understood that the detection strategy is: divide the input image into several pixels, place an anchor point on each pixel, set upper and lower boundaries for three detection heads, and the anchor point is responsible for predicting the target position, and finally outputs the predicted frame with the highest degree of fit with the real frame.

[0071] In one embodiment, pre-training is performed through the following steps to establish a sample database of hanging objects; a corresponding network model is constructed, and a converged model is obtained by training based on a pre-stored data set as a pre-trained model; the pre-trained model is loaded, transfer learning training is performed based on the sample database, and the test data of the pre-trained model in the test set is evaluated using the evaluation criteria of hanging objects; the model corresponding to the optimal test value is saved to obtain the optimal model. It can be understood that it is first necessary to establish a sample database of hanging objects, and manually annotate the pictures in the sample library using the standard pascal voc data format. A target detector is built, a corresponding network model is constructed, and a converged model is obtained by sufficient training based on the COCO data set, and this model is used as a pre-trained model. After obtaining the pre-trained model, the pre-trained model is loaded, transfer learning training is performed based on the established sample library, the network is fine-tuned, and the test data of the model in the test set is evaluated using the evaluation criteria of hanging objects. The model of the optimal test value is saved, and the obtained model is the optimal model, which can be used for algorithm deployment to detect real hanging objects.

[0072] For verification, a series of experiments were designed to integrate the inter-frame differential fusion grayscale projection algorithm with the deep learning model, and field tests were carried out in real environments, including algorithms under different lighting environments. The experimental results show that it performs well in terms of real-time and accuracy, and can capture moving targets on the transmission line in real time, and accurately identify hanging objects through the deep learning model. It not only overcomes the problems of insufficient real-time performance and low recognition efficiency in traditional recognition methods, but also improves the adaptability to complex environments. Under different lighting and weather conditions, it can accurately identify the hanging object area in the image, with strong robustness, which effectively improves the safety monitoring level of transmission lines.

[0073] Focusing on power transmission hanging objects, an important external hidden danger that affects the safe operation of power transmission lines, the paper deeply analyzes the shortcomings of existing recognition algorithms in terms of real-time performance and recognition efficiency, innovatively combines the inter-frame difference method with deep learning, and provides a new solution for the real-time recognition of power transmission hanging objects. It can be put into use in daily actual production work and can be promoted, with good application prospects. The technical effects are as follows:

[0074] The inter-frame difference method for moving target detection is used to identify real-time video streams. It has strong real-time performance and can detect moving targets immediately once they are on site. The deep learning floating object recognition algorithm only recognizes images of moving targets. The algorithm has high operating efficiency and timely and rapid detection, which solves the shortcomings of existing research. The experimental results show that the algorithm proposed in the present invention performs well in a variety of complex environments. It not only has high recognition accuracy, but also has strong real-time performance, which can meet the actual needs of power transmission line safety monitoring. In addition, the algorithm also has good robustness and adaptability, and can cope with recognition tasks under different lighting, angles and occlusion conditions, which is of great significance for improving the safe operation level of power transmission lines.

[0075] The embodiment of the present invention further provides a system for identifying hanging objects on a power line, which is used to implement the method for identifying hanging objects on a power line, comprising:

[0076] An image processing module is used to obtain a real-time video stream and extract a moving target from the real-time video stream to obtain a preliminary screening result; wherein the preliminary screening result at least includes relevant images of the moving target;

[0077] The recognition module is used to input the preliminary screening results into a preset trained recognition model to determine whether the moving target in the preliminary screening results is a hanging object on the transmission line; wherein the recognition model is trained with pre-stored hanging object sample data to identify whether the moving target is a hanging object.

[0078] In one embodiment, the image processing module is specifically used to obtain three consecutive frames of images from a real-time video stream, and calculate the inter-frame difference of the three consecutive frames of images to determine the corresponding image after the frame difference;

[0079] Performing binarization and addition operation on the image after frame difference, and performing grayscale projection on the image after operation to obtain the corresponding grayscale projection curve;

[0080] If the grayscale projection peak value at the target position is greater than the grayscale projection value of the noise point, it is determined that there is a moving target at the target position;

[0081] The inter-frame difference of three consecutive frames of images is calculated according to the following formula:

[0082] G1(x,y)=|F k (x,y)-F k-1 (x,y)|

[0083] G2(x,y)=|F k+1 (x,y)-F k (x,y)|

[0084] Among them, F k-1 (x,y), Fk (x,y), F k+1 (x, y) is the k-1th, kth, k+1th consecutive frame image; G1(x, y), G2(x, y) are the images after frame difference;

[0085] The image after frame difference is binarized and added according to the following formula:

[0086] R(x,y)=G1(x,y)+G2(x,y)

[0087] Among them, R(x,y) is the image after the addition operation.

[0088] The recognition module is specifically used to divide the input related picture of the moving target into a number of pixels; place an anchor point on each pixel, and set multiple detection heads according to the anchor point, and set upper and lower boundaries for the detection heads; the anchor point is responsible for predicting the target position, and finally outputs the predicted frame with the highest fit with the real frame.

[0089] It should be noted that the system described in the above embodiment corresponds to the method described in the above embodiment. Therefore, the undetailed parts of the system described in the above embodiment can be obtained by referring to the contents of the method described in the above embodiment, and will not be repeated here.

[0090] In summary, the implementation of the embodiments of the present invention has the following beneficial effects:

[0091] The method and system for identifying hanging objects on transmission lines provided by the present invention can quickly and accurately identify hanging objects on transmission lines through real-time video analysis based on inter-frame difference fusion deep learning, thereby improving the safe operation and maintenance level of power grids and ensuring stable power supply.

[0092] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for identifying hanging objects on a transmission line, characterized in that: include: Acquire a real-time video stream, and extract a moving target from the real-time video stream to obtain a preliminary screening result; wherein the preliminary screening result at least includes a relevant picture of the moving target; The preliminary screening results are input into a preset trained recognition model to determine whether the moving target in the preliminary screening results is a hanging object on the power transmission line; wherein the recognition model is trained with pre-stored hanging object sample data to identify whether the moving target is a hanging object.

2. The method according to claim 1, characterized in that The extracting of the moving target from the real-time video stream specifically includes: Obtain three consecutive frames of images from a real-time video stream, calculate the inter-frame differences of the three consecutive frames of images, and determine the corresponding images after the frame differences; Performing binarization and addition operation on the image after frame difference, and performing grayscale projection on the image after operation to obtain the corresponding grayscale projection curve; If the grayscale projection peak value at the target position is greater than the grayscale projection value of the noise point, it is determined that there is a moving target at the target position.

3. The method according to claim 2, characterized in that The inter-frame difference of three consecutive frames of images is calculated according to the following formula: G1(x,y)=|F k (x,y)-F k-1 (x,y)| G2(x,y)=|F k+1 (x,y)-F k (x,y)| Among them, F k-1 (x,y), F k (x,y), F k+1 (x, y) is the k-1th, kth, k+1th consecutive frame image; G1(x, y), G2(x, y) are the images after frame difference.

4. The method according to claim 3, characterized in that The image after frame difference is binarized and added according to the following formula: R(x,y)=G1(x,y)+G2(x,y) Among them, R(x, y) is the image after the addition operation.

5. The method according to claim 1, characterized in that The preset trained recognition model at least includes: A feature extraction model is used to extract feature information of an input image according to preset rules; Feature fusion model, used to fuse the extracted feature information to obtain feature values; A detection head model, used to set a corresponding detection head according to the characteristic value; The suppression model is used to extract the prediction boxes and scores of each category whose scores are greater than a preset threshold according to the set detection head, and perform non-maximum suppression using the positions and scores of the prediction boxes.

6. The method according to claim 5, characterized in that The preset trained recognition model is specifically used to divide the input related picture of the moving target into a number of pixels; Place an anchor point on each pixel, set multiple detection heads based on the anchor points, and set upper and lower boundaries for the detection heads; The anchor point is responsible for predicting the target position and ultimately outputting the predicted frame that best matches the true frame.

7. The method according to claim 5, characterized in that Pre-training is performed through the following steps: Establish a sample database of floating objects; Construct the corresponding network model and train it based on the pre-stored data set to obtain a converged model as a pre-training model; Load the pre-trained model, perform transfer learning training based on the sample database, and use the floating object evaluation criteria to evaluate the test data of the pre-trained model in the test set; The model corresponding to the optimal test value is saved to obtain the optimal model.

8. A system for identifying hanging objects on a transmission line, used to implement the method according to any one of claims 1 to 7, characterized in that: include, An image processing module is used to obtain a real-time video stream and extract a moving target from the real-time video stream to obtain a preliminary screening result; wherein the preliminary screening result at least includes relevant images of the moving target; The recognition module is used to input the preliminary screening results into a preset trained recognition model to determine whether the moving target in the preliminary screening results is a hanging object on the transmission line; wherein the recognition model is trained with pre-stored hanging object sample data to identify whether the moving target is a hanging object.

9. The system according to claim 8, characterized in that The image processing module is specifically used to obtain three consecutive frames of images from the real-time video stream, and calculate the inter-frame difference of the three consecutive frames of images to determine the corresponding image after the frame difference; Performing binarization and addition operation on the image after frame difference, and performing grayscale projection on the image after operation to obtain the corresponding grayscale projection curve; If the grayscale projection peak value at the target position is greater than the grayscale projection value of the noise point, it is determined that there is a moving target at the target position; The inter-frame difference of three consecutive frames of images is calculated according to the following formula: G1(x,y)=|F k (x,y)-F k-1 (x,y)| G2(x,y)=|F k+1 (x,y)-F k (x,y)| Among them, F k-1 (x,y), F k (x,y), F k+1 (x, y) is the k-1th, kth, k+1th consecutive frame image; G1(x, y), G2(x, y) are the images after frame difference; The image after frame difference is binarized and added according to the following formula: R(x,y)=G1(x,y)+G2(x,y) Among them, R(x, y) is the image after the addition operation.

10. The system according to claim 9, characterized in that The recognition module is specifically used to divide the input related picture of the moving target into a number of pixels; Place an anchor point on each pixel, set multiple detection heads based on the anchor points, and set upper and lower boundaries for the detection heads; The anchor point is responsible for predicting the target position and ultimately outputting the predicted frame that best matches the true frame.

Images