Multi-rotor unmanned aerial vehicle electric power facility monitoring system based on real-time video analysis

Abstract

The invention provides a multi-rotor unmanned aerial vehicle electric power facility monitoring system based on real-time video analysis. The system comprises a flight control module, a video acquisition and analysis module, a monitoring and analysis module and a response module. The flight control module is used for controlling the unmanned aerial vehicle to carry out power monitoring inspection navigation along a preset path; the video acquisition and analysis module collects real-time video data of the electric power facility through a high-definition camera carried by the unmanned aerial vehicle; the monitoring analysis module is used for analyzing the real-time video data of the electric power facility to identify the abnormal condition of the electric power facility; the response module is used for marking an abnormal area and sending an alarm notification to related maintenance personnel when the abnormal condition of the electric power facility is identified; according to the invention, while the video stream data of the electric power facility can be obtained with high quality in the flight process of the unmanned aerial vehicle, accurate electric power facility anomaly analysis is provided, and powerful data support is provided for an electric power facility management department.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle monitoring systems, and in particular to a multi-rotor unmanned aerial vehicle power facility monitoring system based on real-time video analysis. Background Art

[0002] In the current management of power facilities, monitoring and maintaining the condition of transmission lines and their components is crucial; traditional ground inspection methods are not only time-consuming and labor-intensive, but also difficult to achieve the desired monitoring effect in some cases, especially in areas with complex terrain or difficult to access; with the advancement of technology, drone technology has been introduced for monitoring power infrastructure. Using drones for real-time video monitoring can not only improve the coverage and frequency of inspections, but also capture detailed images of key facilities through high-definition cameras, providing strong data support for fault prevention and timely maintenance; however, how to quickly and accurately identify and analyze potential abnormal conditions from large amounts of real-time video data remains a technical challenge.

[0003] According to the related disclosed technical solutions, the technology with the publication number CN115471464A proposes a method for automatically identifying defects in power poles and towers based on drone aerial photography, which includes the following steps: S1: Check whether the functions of the drone used are intact, and test the drone to ensure that the conditions for aerial photography are met; S2: Draw the coordinates of the power poles and towers of the transmission line, and enter the coordinates of the power poles and towers and the description information of the power poles and towers into the inspection system; S3: Perform drone aerial photography according to the drawn coordinates of the power poles and towers of the transmission line; This method for automatically identifying defects in power poles and towers based on drone aerial photography, by establishing a tower defect detection model, using drones to conduct aerial photography inspections, and transmitting drone inspection video images to the back-end server, the solution realizes the status monitoring of power pole and node damage identification and diagnosis, and identifies and warns the defects of power poles and nodes, thereby stably detecting the safety status of power facilities and ensuring the safe operation of the line; However, the solution lacks high-quality collection of real-time video stream data and review and integration of detection results, which may affect the accuracy and reliability of identification and diagnosis due to the different quality of the acquired aerial photography data or insufficient data processing. Summary of the invention

[0004] The purpose of the present invention is to propose a multi-rotor UAV power facility monitoring system based on real-time video analysis to address the current deficiencies.

[0005] The present invention adopts the following technical solution:

[0006] A multi-rotor UAV power facility monitoring system based on real-time video analysis, the system comprising a flight control module, a video acquisition and analysis module, a monitoring and analysis module, and a response module;

[0007] The flight control module is used to control the UAV to perform power monitoring and inspection navigation along a preset path; the video acquisition and analysis module collects real-time video data of power facilities through the high-definition camera carried by the UAV; the monitoring and analysis module is used to analyze the real-time video data of power facilities to identify abnormal conditions of power facilities; the response module is used to mark the abnormal area and send an alarm notification to relevant maintenance personnel when an abnormal condition of the power facility is identified;

[0008] The flight control module includes a path setting unit, a flight execution unit and a safety monitoring unit; the path setting unit is used to receive the UAV flight path set by the user, and the UAV flight path includes a starting point, a flight route, a video acquisition point on the flight route and a destination; the flight execution unit is used to control the UAV to perform specific flight operations according to the preset UAV flight path; the safety monitoring unit is used to monitor the flight environment of the UAV during flight, and automatically adjust the flight path to avoid risk obstacles when potential risks are detected in the flight environment;

[0009] Furthermore, the video acquisition and analysis module includes a target recognition unit, a video acquisition unit and a video stabilization unit; the target recognition unit is used to adjust the camera to ensure that the target acquisition area is always within the camera's acquisition field of view when the drone arrives at the video acquisition point, and the target acquisition area is the area where the power facilities to be monitored are located; the video acquisition unit is used to collect real-time power facility video data through a high-definition camera, and the video stabilization unit is used to stabilize the video content captured by the camera to reduce the image jitter and blur caused by the movement of the drone;

[0010] Further, the monitoring and analysis module includes an image capture unit and an abnormality analysis unit, wherein the image capture unit is used to capture an image of the power facility from the collected real-time video data of the power facility, and the abnormality analysis unit is used to analyze the image of the power facility to obtain abnormal conditions of the power facility;

[0011] Furthermore, the specific working process of the target recognition unit is as follows:

[0012] S21: When the UAV reaches each video collection point, the camera focal length is set to the shortest adjustable focal length;

[0013] S22: Identify the target acquisition area within the camera field of view through image recognition technology, and set a plurality of feature points at the edge of the target acquisition area; the feature points are extreme values ​​or inflection points at the edge of the target acquisition area;

[0014] S23: Obtaining the geometric center of the target acquisition area according to the feature points, and adjusting the yaw angle and pitch angle of the camera so that the center position of the camera acquisition field of view is aligned with the geometric center;

[0015] S24: gradually increasing the focal length of the camera, while ensuring that the camera image is clear, so that the target acquisition area can always be acquired within the camera's field of view;

[0016] Furthermore, the image capture unit captures multiple frames of power facility images from the real-time video stream corresponding to each video acquisition point, and selects the multiple frames of power facility images for analysis and detection according to the image quality of the captured multiple frames of power facility images;

[0017] Furthermore, the specific process of the abnormal analysis unit for analyzing abnormal conditions of power facilities is as follows:

[0018] S41: for multiple frames of power facility images selected from a certain video acquisition point, input them into a pre-trained anomaly recognition model, and obtain the output of the anomaly recognition model, where the output of the anomaly recognition model is the degree of anomaly corresponding to the video acquisition point;

[0019] S42: Cluster all the abnormality data obtained in the previous step to generate m clusters, each cluster represents a group of data points with similar abnormality data; calculate the comprehensive abnormality value P of the video acquisition point:

[0020]

[0021] Among them, β j is the aggregation weight of the jth cluster, L j is the mean of the abnormality degree within the jth cluster;

[0022] S43: setting multiple abnormal response threshold intervals. When the comprehensive abnormal value P falls within a higher abnormal response threshold interval, it indicates that the abnormality of the power facility corresponding to the video acquisition point is more serious.

[0023] S44: performing the above operation for the multiple frames of power facility images screened out at each video acquisition point, thereby obtaining an overall analysis and evaluation of abnormal conditions of the power facilities on the flight path of the drone;

[0024] A multi-rotor UAV power facility monitoring method based on real-time video analysis, the method comprising the following steps:

[0025] S1: Setting a UAV flight path, wherein the UAV flight path includes a starting point, a flight route, a video collection point on the flight route, and a destination;

[0026] S2: Control the drone to fly along a preset flight path. When the drone reaches a designated video collection point, control the drone to capture real-time video data of the power facility through a high-definition camera.

[0027] S3: Capture high-quality images of power facilities from real-time video data;

[0028] S4: Evaluate the abnormal conditions of the power facilities corresponding to each video collection point based on the power facility images;

[0029] S5: Send an alarm notification to relevant maintenance personnel when an abnormal situation occurs in the power facility.

[0030] The beneficial effects achieved by the present invention are:

[0031] The present invention uses a target recognition unit to make real-time adjustments to the camera during drone shooting, thereby ensuring that the key monitoring area is always within the camera's optimal field of view, ensuring the clarity and integrity of the captured video image; the abnormality analysis unit clusters and integrates the acquired abnormality degree data for analysis, thereby improving the efficiency and accuracy of fault diagnosis and making the monitoring and maintenance of power facilities more efficient and systematic. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily drawn to scale, but the emphasis is placed on illustrating the principles of the embodiments. In different views, the same reference numerals designate corresponding parts.

[0033] Figure 1 It is a schematic diagram of the overall module of the present invention.

[0034] Figure 2 The present invention is a flow chart of a multi-rotor UAV power facility monitoring method based on real-time video analysis.

[0035] Figure 3 Schematic diagram of the work flow of the target identification unit of the present invention.

[0036] Figure 4 Schematic diagram of the work flow of the abnormality analysis unit of the present invention. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with its embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention; for those skilled in the art, other systems, methods and / or features of the present embodiment will become apparent after reviewing the following detailed description; it is intended that all such additional systems, methods, features and advantages are included in this specification; included within the scope of the present invention and protected by the appended claims; additional features of the disclosed embodiments are described in the following detailed description, and these features will be apparent from the following detailed description.

[0038] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the drawings, it is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or component referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limitations on this patent. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0039] Embodiment 1:

[0040] like Figure 1 As shown, this embodiment provides a multi-rotor UAV power facility monitoring system based on real-time video analysis, the system includes a flight control module, a video acquisition and analysis module, a monitoring and analysis module and a response module;

[0041] The flight control module is used to control the UAV to perform power monitoring and inspection navigation along a preset path; the video acquisition and analysis module collects real-time video data of power facilities through the high-definition camera carried by the UAV; the monitoring and analysis module is used to analyze the real-time video data of power facilities to identify abnormal conditions of power facilities; the response module is used to mark the abnormal area and send an alarm notification to relevant maintenance personnel when an abnormal condition of the power facility is identified;

[0042] The flight control module includes a path setting unit, a flight execution unit and a safety monitoring unit; the path setting unit is used to receive the UAV flight path set by the user, and the UAV flight path includes a starting point, a flight route, a video acquisition point on the flight route and a destination; the flight execution unit is used to control the UAV to perform specific flight operations according to the preset UAV flight path; the safety monitoring unit is used to monitor the flight environment of the UAV during flight, and automatically adjust the flight path to avoid risk obstacles when potential risks are detected in the flight environment;

[0043] Furthermore, the video acquisition and analysis module includes a target recognition unit, a video acquisition unit and a video stabilization unit; the target recognition unit is used to adjust the camera to ensure that the target acquisition area is always within the camera's acquisition field of view when the drone arrives at the video acquisition point, and the target acquisition area is the area where the power facilities to be monitored are located; the video acquisition unit is used to collect real-time power facility video data through a high-definition camera, and the video stabilization unit is used to stabilize the video content captured by the camera to reduce the image jitter and blur caused by the movement of the drone;

[0044] Further, the monitoring and analysis module includes an image capture unit and an abnormality analysis unit, wherein the image capture unit is used to capture an image of the power facility from the collected real-time video data of the power facility, and the abnormality analysis unit is used to analyze the image of the power facility to obtain abnormal conditions of the power facility;

[0045] Further, such as Figure 3 As shown, the specific working process of the target recognition unit is as follows:

[0046] S21: When the UAV reaches each video collection point, the camera focal length is set to the shortest adjustable focal length;

[0047] S22: Identify the target acquisition area within the camera field of view through image recognition technology, and set a plurality of feature points at the edge of the target acquisition area; the feature points are extreme values ​​or inflection points at the edge of the target acquisition area;

[0048] S23: Obtaining the geometric center of the target acquisition area according to the feature points, and adjusting the yaw angle and pitch angle of the camera so that the center position of the camera acquisition field of view is aligned with the geometric center;

[0049] S24: gradually increasing the focal length of the camera, while ensuring that the camera image is clear, so that the target acquisition area can always be acquired within the camera's field of view;

[0050] Furthermore, in the process of increasing the camera focal length in step S24, the focal length adjustment will continue until it increases to a maximum focal length that satisfies the following formula:

[0051]

[0052] Where f is the focal length after growth adjustment, X is the maximum horizontal distance of the target acquisition area in the camera field of view, and Y is the maximum vertical distance of the target acquisition area in the camera field of view. X and Y are obtained by the maximum horizontal and vertical spacing between the feature points; θ is the field of view of the camera. During the adjustment of the focal length, θ≤θ must always be satisfied. max ,θ max is the maximum allowable field of view, which is preset according to the technical specifications of the camera and the actual monitoring environment; α is the anti-shake coefficient, and its value range is [0.1, 0.2];

[0053] By gradually increasing the focal length from the shortest focal length of the camera, the ideal focal length is found to cover the target acquisition area, allowing the camera to flexibly respond to different monitoring conditions, ensuring that the video image of the target acquisition area can be fully acquired while providing maximum video image detail capture; by aligning the center of the camera's acquisition field of view with the geometric center of the target area, it helps to improve the accuracy of image analysis; in the process of adjusting the focal length, the possible flight jitter of the drone is fully considered. Even if the drone jitters, the complete video image information of the target acquisition area can still be acquired, thereby providing stable and continuous monitoring data and providing high-quality data support for the subsequent monitoring process.

[0054] Embodiment 2:

[0055] This embodiment should be understood to include at least all the features of any of the above embodiments, and further improve upon them;

[0056] This embodiment provides a multi-rotor UAV power facility monitoring system based on real-time video analysis, the system includes a flight control module, a video acquisition and analysis module, a monitoring and analysis module, and a response module;

[0057] The flight control module is used to control the UAV to perform power monitoring and inspection navigation along a preset path; the video acquisition and analysis module collects real-time video data of power facilities through the high-definition camera carried by the UAV; the monitoring and analysis module is used to analyze the real-time video data of power facilities to identify abnormal conditions of power facilities; the response module is used to mark the abnormal area and send an alarm notification to relevant maintenance personnel when an abnormal condition of the power facility is identified;

[0058] The flight control module includes a path setting unit, a flight execution unit and a safety monitoring unit; the path setting unit is used to receive the UAV flight path set by the user, and the UAV flight path includes a starting point, a flight route, a video acquisition point on the flight route and a destination; the flight execution unit is used to control the UAV to perform specific flight operations according to the preset UAV flight path; the safety monitoring unit is used to monitor the flight environment of the UAV during flight, and automatically adjust the flight path to avoid risk obstacles when potential risks are detected in the flight environment;

[0059] Further, the monitoring and analysis module includes an image capture unit and an abnormality analysis unit, wherein the image capture unit is used to capture an image of the power facility from the collected real-time video data of the power facility, and the abnormality analysis unit is used to analyze the image of the power facility to obtain abnormal conditions of the power facility;

[0060] Furthermore, the image capture unit captures multiple frames of power facility images in the real-time video stream corresponding to each video acquisition point, and selects multiple frames of power facility images for analysis and detection according to the image quality of the captured multiple frames of power facility images; the screening of the power facility images can be completed by clarity detection, contrast evaluation and exposure level analysis in the prior art, which will not be elaborated here;

[0061] Further, such as Figure 4 As shown, the specific process of the abnormal analysis unit for analyzing abnormal conditions of power facilities is as follows:

[0062] S41: for multiple frames of power facility images selected from a certain video acquisition point, input them into a pre-trained anomaly recognition model, and obtain the output of the anomaly recognition model, where the output of the anomaly recognition model is the degree of anomaly corresponding to the video acquisition point;

[0063] S42: Cluster all the abnormality data obtained in the previous step to generate m clusters, each cluster represents a group of data points with similar abnormality data; calculate the comprehensive abnormality value P of the video acquisition point:

[0064]

[0065] Among them, β j is the aggregation weight of the jth cluster, L j is the mean of the abnormality degree within the jth cluster;

[0066] S43: setting multiple abnormal response threshold intervals. When the comprehensive abnormal value P falls within a higher abnormal response threshold interval, it indicates that the abnormality of the power facility corresponding to the video acquisition point is more serious.

[0067] S44: performing the above operation for the multiple frames of power facility images screened out at each video acquisition point, thereby obtaining an overall analysis and evaluation of abnormal conditions of the power facilities on the flight path of the drone;

[0068] Furthermore, the abnormality recognition model used in step S41 can be obtained through deep learning algorithm training in the prior art;

[0069] Furthermore, in step S42, the aggregation weight β j satisfy:

[0070]

[0071] Among them, n j is the total number of abnormal degree data in the jth cluster;

[0072] Furthermore, in step S42, the L j satisfy:

[0073]

[0074] Among them, K i,j is the i-th abnormality data value in the j-th cluster;

[0075] By using high-quality image input to perform abnormal analysis of power facilities, the accuracy of model prediction is ensured, and misidentification caused by blurred images, improper exposure or other quality problems is reduced; by assigning data points with similar abnormality levels to the same cluster and assigning a weight based on the amount of data to each cluster, it can ensure that large sample clusters have a greater influence on the final analysis results, reducing errors caused by small data volumes or accidental abnormal data, making the final abnormal analysis results more reliable and representative, providing strong data support and decision-making basis for power facility management departments;

[0076] Furthermore, the response module provides different degrees of alarm notifications to relevant maintenance personnel according to the abnormal response threshold interval of the comprehensive abnormal value of each video acquisition point; the specific examples are as follows:

[0077] Assume that γ1, γ2 and γ3 are the boundary values ​​of each preset abnormal response threshold interval, and satisfy γ1<γ2<γ3, then:

[0078] When γ1≤P<γ2, a primary alarm notification is issued to the maintenance personnel to remind them to conduct a preliminary inspection or increase the monitoring frequency, and the location of the power facility at the video collection point corresponding to the comprehensive abnormal value is provided to the maintenance personnel;

[0079] When γ2≤P<γ3, a medium-level alarm notification is issued to the maintenance personnel to remind them to conduct detailed inspections and perform necessary preventive maintenance, and the location of the power facility at the video collection point corresponding to the comprehensive abnormal value is provided to the maintenance personnel;

[0080] When P ≥ γ3, a high-level alarm notification is issued to the maintenance personnel to remind them to execute emergency response plan measures to prevent the safe operation of the power facilities from being affected, and the location of the power facilities at the video collection point corresponding to the comprehensive abnormal value is provided to the maintenance personnel;

[0081] like Figure 2 As shown, this embodiment provides a multi-rotor UAV power facility monitoring method based on real-time video analysis, and the method includes the following steps:

[0082] S1: Setting a UAV flight path, wherein the UAV flight path includes a starting point, a flight route, a video collection point on the flight route, and a destination;

[0083] S2: Control the drone to fly along a preset flight path. When the drone reaches a designated video collection point, control the drone to capture real-time video data of the power facility through a high-definition camera.

[0084] S3: Capture high-quality images of power facilities from real-time video data;

[0085] S4: Evaluate the abnormal conditions of the power facilities corresponding to each video collection point based on the power facility images;

[0086] S5: Send an alarm notification to relevant maintenance personnel when an abnormal situation occurs in the power facility.

[0087] The contents disclosed above are only preferred feasible embodiments of the present invention, and do not limit the protection scope of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention specification and drawings are included in the protection scope of the present invention. In addition, the elements therein can be updated as technology develops.

Claims

1. A multi-rotor UAV power facility monitoring system based on real-time video analysis, characterized in that: The system includes a flight control module, a video acquisition and analysis module, a monitoring and analysis module, and a response module; The flight control module is used to control the UAV to perform power monitoring and inspection navigation along a preset path; the video acquisition and analysis module collects real-time video data of power facilities through the high-definition camera carried by the UAV; the monitoring and analysis module is used to analyze the real-time video data of power facilities to identify abnormal conditions of power facilities; the response module is used to mark the abnormal area and send an alarm notification to relevant maintenance personnel when an abnormal condition of the power facility is identified; The flight control module includes a path setting unit, a flight execution unit and a safety monitoring unit; the path setting unit is used to receive the UAV flight path set by the user, and the UAV flight path includes a starting point, a flight route, a video acquisition point on the flight route and a destination; the flight execution unit is used to control the UAV to perform specific flight operations according to the preset UAV flight path; the safety monitoring unit is used to monitor the flight environment of the UAV during flight, and automatically adjust the flight path to avoid risk obstacles when potential risks are detected in the flight environment.

2. A multi-rotor UAV power facility monitoring system based on real-time video analysis according to claim 1, characterized in that: The video acquisition and analysis module includes a target recognition unit, a video acquisition unit and a video stabilization unit; the target recognition unit is used to adjust the camera to ensure that the target acquisition area is always within the camera's acquisition field of view when the drone arrives at the video acquisition point, and the target acquisition area is the area where the power facilities to be monitored are located; the video acquisition unit is used to collect real-time power facility video data through a high-definition camera, and the video stabilization unit is used to stabilize the video content captured by the camera to reduce image jitter and blur caused by drone movement.

3. The multi-rotor UAV power facility monitoring system based on real-time video analysis according to claim 1 is characterized in that: The monitoring and analysis module includes an image capture unit and an abnormality analysis unit. The image capture unit is used to capture images of power facilities from collected real-time video data of power facilities, and the abnormality analysis unit is used to analyze the images of power facilities to obtain abnormal conditions of the power facilities.

4. The multi-rotor UAV power facility monitoring system based on real-time video analysis according to claim 1 is characterized in that: The specific workflow of the target recognition unit is as follows: S21: When the UAV reaches each video collection point, the camera focal length is set to the shortest adjustable focal length; S22: Identify the target acquisition area within the camera field of view through image recognition technology, and set a plurality of feature points at the edge of the target acquisition area; the feature points are extreme values ​​or inflection points at the edge of the target acquisition area; S23: Obtaining the geometric center of the target acquisition area according to the feature points, and adjusting the yaw angle and pitch angle of the camera so that the center position of the camera acquisition field of view is aligned with the geometric center; S24: gradually increase the focal length of the camera, so that the target acquisition area can always be acquired within the field of view of the camera while ensuring the clarity of the camera image.

5. The multi-rotor UAV power facility monitoring system based on real-time video analysis according to claim 1 is characterized in that: The image capture unit captures multiple frames of power facility images from the real-time video stream corresponding to each video acquisition point, and selects multiple frames of power facility images for analysis and detection according to the image quality of the captured multiple frames of power facility images.

6. The multi-rotor UAV power facility monitoring system based on real-time video analysis according to claim 1 is characterized in that: The specific process of the abnormal analysis unit for analyzing abnormal conditions of power facilities is as follows: S41: for multiple frames of power facility images selected from a certain video acquisition point, input them into a pre-trained anomaly recognition model, and obtain the output of the anomaly recognition model, where the output of the anomaly recognition model is the degree of anomaly corresponding to the video acquisition point; S42: clustering all the abnormality degree data obtained in the previous step to generate m clusters, each cluster represents a group of data points with similar abnormality degree data; Calculate the comprehensive outlier value P of the video acquisition point: Among them, β j is the aggregation weight of the jth cluster, L j is the mean of the abnormality degree within the jth cluster; S43: setting multiple abnormal response threshold intervals. When the comprehensive abnormal value P falls within a higher abnormal response threshold interval, it indicates that the abnormality of the power facility corresponding to the video acquisition point is more serious. S44: The above operation is performed on the multiple frames of power facility images screened out at each video acquisition point, so as to obtain an overall analysis and evaluation of abnormal conditions of the power facilities on the UAV flight path.

7. A multi-rotor UAV power facility monitoring method based on real-time video analysis, characterized in that: The method comprises the following steps: S1: Setting a UAV flight path, wherein the UAV flight path includes a starting point, a flight route, a video collection point on the flight route, and a destination; S2: Control the drone to fly along a preset flight path. When the drone reaches a designated video collection point, control the drone to capture real-time video data of the power facility through a high-definition camera. S3: Capture high-quality images of power facilities from real-time video data; S4: Evaluate the abnormal conditions of the power facilities corresponding to each video collection point based on the power facility images; S5: Send an alarm notification to relevant maintenance personnel when an abnormal situation occurs in the power facility.

Images