Unmanned aerial vehicle inspection method and device, unmanned aerial vehicle, storage medium and program product

Abstract

The invention provides an unmanned aerial vehicle inspection method and device, an unmanned aerial vehicle, a storage medium and a program product, and relates to the technical field of unmanned aerial vehicles, and the method comprises the steps: obtaining to-be-inspected task information and historical fault data corresponding to the to-be-inspected task information; determining a first inspection plan according to the to-be-inspected task information and the historical fault data; executing an inspection task according to the first inspection plan, and obtaining real-time inspection data; when a suspected abnormal condition is detected according to the real-time inspection data, adjusting the first inspection plan to obtain a second inspection plan; and continuing to execute the inspection task according to the second inspection plan. The problem that in the prior art, the unmanned aerial vehicle inspection efficiency and the intelligent level are low can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicles, and in particular to an unmanned aerial vehicle inspection method and device, an unmanned aerial vehicle, a storage medium and a program product. BACKGROUND

[0002] At present, unmanned aerial vehicle inspection has been widely used in the field of infrastructure inspection such as power, oil pipeline and railway. In the prior art, a typical unmanned aerial vehicle inspection scheme usually performs an inspection task based on a pre-defined inspection standard, and collects data of a target facility through various sensors (such as a camera, an infrared thermal imager, etc.) carried. Specifically, in the power inspection scenario, the unmanned aerial vehicle flies along the power transmission line, and uses a high-definition camera to shoot images of each line and tower according to pre-set parameters; in the oil pipeline inspection scenario, the unmanned aerial vehicle flies along the pipeline laying line, and detects the surface temperature of each pipeline at a pre-set position through an infrared thermal imager to identify potential leakage points.

[0003] However, the above inspection method based on a unified inspection standard is difficult to dynamically adjust the inspection focus according to the real-time state of the target facility or a specific risk area. This kind of inspection method lacking flexibility and pertinence results in low inspection efficiency. Therefore, how to improve the efficiency and intelligent level of unmanned aerial vehicle inspection is a problem to be solved at present. SUMMARY

[0004] The present application provides an unmanned aerial vehicle inspection method, device, unmanned aerial vehicle, storage medium and program product to solve the problem of low efficiency and intelligent level of unmanned aerial vehicle inspection in the prior art.

[0005] The present application provides an unmanned aerial vehicle inspection method, comprising: obtaining task information to be inspected and corresponding historical fault data thereof; determining a first inspection plan according to the task information to be inspected and the historical fault data; performing an inspection task according to the first inspection plan and obtaining real-time inspection data; adjusting the first inspection plan to obtain a second inspection plan when a suspected abnormal condition is detected according to the real-time inspection data; continuing to perform the inspection task according to the second inspection plan.

[0006] According to the unmanned aerial vehicle inspection method provided by the present application, after the first inspection plan is adjusted to obtain a second inspection plan when a suspected abnormal condition is detected according to the real-time inspection data, the method further comprises: obtaining a current task progress; sending the current task progress and the second inspection plan to a target unmanned aerial vehicle through ad hoc network communication. receiving a third inspection plan fed back by the target UAV; continuing to perform the inspection task according to the second inspection plan, including: continuing to perform the inspection task according to the third inspection plan.

[0007] According to the unmanned aerial vehicle inspection method provided by the application, the to-be-inspected task information includes a facility type and a facility position of a to-be-inspected facility, the first inspection plan is determined according to the to-be-inspected task information and the historical failure data, and the first inspection plan includes: a facility potential failure is predicted according to the facility type and the historical failure data by using a first failure prediction model, and a facility failure prediction result is obtained; a failure area is predicted according to the facility position and the historical failure data by using a second failure prediction model, and a failure area prediction result is obtained; an inspection parameter is determined according to the facility failure prediction result, and a key inspection area is determined according to the failure area prediction result; The first inspection plan includes the inspection parameter and the key inspection area.

[0008] According to the unmanned aerial vehicle inspection method provided by the application, the first inspection plan is used to perform an inspection task and obtain real-time inspection data, and the method includes: an initial flight plan is obtained; the inspection task is performed according to the first inspection plan and the initial flight plan, and real-time inspection data and real-time environment data are obtained; after the inspection task is performed according to the first inspection plan and the real-time inspection data are obtained, the method further includes: the initial flight plan is adjusted according to the real-time environment data, and an adjusted flight plan is obtained; the second inspection plan is used to continue to perform the inspection task, and the method includes: the inspection task is continued to be performed according to the second inspection plan and the adjusted flight plan.

[0009] According to the unmanned aerial vehicle inspection method provided by the application, the initial flight plan includes an initial flight path and an initial flight parameter, and the real-time environment data includes three-dimensional point cloud data, obstacle distance, air pressure height, and weather data. the initial flight plan is adjusted according to the real-time environment data, and an adjusted flight plan is obtained, and the method includes: a real-time three-dimensional model of a flight environment is constructed according to the three-dimensional point cloud data and the obstacle distance; According to the real-time three-dimensional model and the barometric altitude, real-time terrain and obstacle information are identified; According to the real-time terrain and the obstacle information, the initial flight path is adjusted to obtain an adjusted flight path; According to the weather data, the initial flight parameters are adjusted to obtain adjusted flight parameters; The adjusted flight plan includes the adjusted flight path and the adjusted flight parameters.

[0010] According to the present application, a UAV inspection method is provided, and after the real-time inspection data is obtained, the method further includes: When a suspected abnormal condition is detected according to the real-time inspection data, the real-time inspection data corresponding to the suspected abnormal condition is sent to a control end for analysis and processing by the control end.

[0011] The present application further provides a UAV inspection device, which includes: A first obtaining module is configured to obtain task information to be inspected and corresponding historical fault data thereof; A determining module is configured to determine a first inspection plan according to the task information to be inspected and the historical fault data; A second obtaining module is configured to execute an inspection task according to the first inspection plan and obtain real-time inspection data; An adjusting module is configured to adjust the first inspection plan to obtain a second inspection plan when a suspected abnormal condition is detected according to the real-time inspection data; An executing module is configured to continue to execute the inspection task according to the second inspection plan.

[0012] The present application further provides a UAV, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the UAV inspection method of any one of the above.

[0013] The present application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the UAV inspection method of any one of the above.

[0014] The present application further provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the UAV inspection method of any one of the above.

[0015] The unmanned aerial vehicle inspection method and device, unmanned aerial vehicle, storage medium and program product provided by the application obtain task information to be inspected and corresponding historical fault data, then determine a first inspection plan according to the task information to be inspected and the historical fault data, further execute an inspection task according to the first inspection plan and obtain real-time inspection data, adjust the first inspection plan when detecting a suspected abnormal condition according to the real-time inspection data to obtain a second inspection plan, and continue to execute the inspection task according to the second inspection plan. In the application, the first inspection plan is formulated based on the task information to be inspected and combined with the historical fault data to determine a key inspection area and an inspection parameter, further, the real-time inspection data is analyzed during the execution of the inspection task, the first inspection plan is adjusted when a suspected abnormal condition is detected, and then the second inspection plan obtained by the adjustment is used to continue to execute the task. The inspection plan is dynamically adjusted according to the abnormal detection result detected by the real-time inspection data, which can make the inspection work more targeted, avoid subsequent re-inspection, and thus improve the efficiency and intelligent level of the unmanned aerial vehicle inspection. In addition, the real-time inspection data is preliminarily detected locally on the unmanned aerial vehicle, which can reduce the subsequent data processing amount of the human or control end, and thus further improve the overall inspection efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0017] Figure 1 is one of the flowcharts of the unmanned aerial vehicle inspection method provided by the application; Figure 2 is another flowchart of the unmanned aerial vehicle inspection method provided by the application; Figure 3 is a structural schematic diagram of the unmanned aerial vehicle inspection device provided by the application; Figure 4 is a structural schematic diagram of the unmanned aerial vehicle provided by the application. DETAILED DESCRIPTION

[0018] In order to make the objects, technical solutions and advantages of the application clearer, the technical solutions in the application will be described clearly and completely in the following with reference to the drawings in the application. Obviously, the described embodiments are some embodiments of the application, but not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative effort belong to the protection scope of the application.

[0019] Drone inspections are now widely used in infrastructure inspections, including power plants, oil pipelines, and railways. Typical drone inspection solutions typically perform inspections based on pre-defined inspection standards while simultaneously collecting data from target facilities using various onboard sensors (such as cameras and infrared thermal imagers). Specifically, in power plant inspections, drones fly along transmission lines, using high-definition cameras to capture images of the lines and towers according to preset parameters. In oil pipeline inspections, drones fly along pipeline routes, using infrared thermal imagers to measure surface temperatures at pre-set locations on the pipelines to identify potential leaks.

[0020] However, this inspection method, based on standardized inspection standards, makes it difficult to dynamically adjust inspection priorities based on the real-time status of target facilities or specific risk areas. This lack of flexibility and targeted approach results in low inspection efficiency. Therefore, improving the efficiency and intelligence of drone inspections is a pressing issue.

[0021] Based on the above problems, the present invention proposes a drone inspection method, device, drone, storage medium and program product. Figures 1-4 Provide a description.

[0022] Figure 1 This is one of the flow charts of the drone inspection method provided by the present invention, such as Figure 1 As shown, the drone inspection method includes: step S110, step S120, step S130, step S140 and step S150.

[0023] Step S110: Obtain information of tasks to be inspected and their corresponding historical fault data.

[0024] In this embodiment, the drone inspection method is applied to a drone.

[0025] Considering the inspection method based on unified inspection standards in the existing technology, it is impossible to flexibly adjust the inspection focus according to the actual conditions of the facilities. For some key parts or areas where hidden dangers may exist, it is impossible to increase the inspection frequency and obtain multi-dimensional inspection data in a timely manner. When the inspection is completed, the control end or the analyst analyzes the inspection data and finds anomalies, and then needs to re-inspect for the abnormal situation, resulting in low inspection efficiency. At the same time, all inspection data are manually analyzed after the inspection is completed. Faced with a large amount of collected inspection data, manual analysis is time-consuming and labor-intensive, with low overall efficiency and prone to omissions. Based on the above, in this embodiment, based on the information of the task to be inspected, combined with historical fault data, a first inspection plan is formulated to determine the key inspection areas and inspection parameters. Furthermore, in the process of executing the inspection task, the real-time inspection data is analyzed. When a suspected abnormal situation is detected, the first inspection plan is adjusted, and then the task is continued based on the adjusted second inspection plan. By dynamically adjusting inspection plans based on anomaly detection results from real-time inspection data, inspections can be conducted more effectively and efficiently, avoiding the need for subsequent re-inspections. This improves the efficiency and intelligence of drone inspections. Furthermore, by performing preliminary checks on real-time inspection data locally on the drone, the amount of subsequent data processing required by humans or the control end can be reduced, further improving overall inspection efficiency.

[0026] Here, the pending inspection mission information refers to information related to the inspection mission to be performed by the drone, including but not limited to: the facility type, location, and number of the facility to be inspected, as well as the inspection purpose, mission start point, mission end point, and mission time. Facility types include but are not limited to photovoltaic panels, towers, and oil pipelines; facility locations include but are not limited to GPS (Global Positioning System) coordinates and region tags; and historical fault data refers to data related to historical faults that have occurred in the facility to be inspected, including but not limited to the fault area, fault type, fault severity, and fault occurrence time.

[0027] Step S120: determining a first inspection plan according to the information of the inspection task to be inspected and the historical fault data.

[0028] As an implementation method, historical fault data can be statistically analyzed to obtain statistical analysis results, wherein the statistical analysis results include historical fault types and their occurrence probabilities; then, based on the inspection task information and the statistical analysis results, the key inspection areas and inspection parameters are determined to obtain the first inspection plan.

[0029] As a further implementation, the to-be-inspected task information includes facility information and facility type of the to-be-inspected facility. The facility potential fault can be predicted according to the facility type and historical fault data through a first fault prediction model to obtain a facility fault prediction result, and the fault area can be predicted according to the facility location and historical fault data through a second fault prediction model to obtain a fault area prediction result. Then, the inspection parameters are determined according to the facility fault prediction result, and the key inspection area is determined according to the fault area prediction result. The first inspection plan includes the inspection parameters and the key inspection area. The specific implementation process can refer to the following embodiments, which will not be repeated here. Compared with the previous implementation, the inspection plan formulated by the present implementation is more accurate, thereby reducing the probability of repeated inspection and improving the inspection efficiency.

[0030] It should be noted that the key inspection area is the area or part that needs to be inspected. For example, if the power inspection scene is used to inspect the power transmission line, the key inspection area includes but is not limited to: insulator string, hardware connection point, wire joint, tower foundation. The inspection parameters are some parameters in the inspection task execution process, including but not limited to: flight height, flight speed, data acquisition parameters. In addition, it should be understood that the first inspection plan can include other information in addition to the key inspection area and the inspection parameters, for example, the inspection time, which can be determined according to conventional methods.

[0031] Step S130, according to the first inspection plan, the inspection task is executed, and real-time inspection data is obtained.

[0032] After the first inspection plan is formulated, the inspection task is executed according to the first inspection plan. And in the process of executing the inspection task, the inspection data is obtained in real time, which is recorded as real-time inspection data.

[0033] Step S140, when a suspected abnormal situation is detected according to the real-time inspection data, the first inspection plan is adjusted to obtain a second inspection plan.

[0034] After obtaining the real-time inspection data, the real-time inspection data is preliminarily detected. The corresponding detection rule can be determined according to the to-be-inspected task information. The detection rule is pre-set, and different detection rules can be set according to different types of inspection facilities and different inspection purposes. The specific rule can be set according to actual needs, which is not limited here.

[0035] It should be noted that considering the computing power and real-time performance of the unmanned aerial vehicle, the detection rule can be set as a relatively basic and fast rule such as threshold detection. The purpose is to only detect a suspected abnormal situation, so as not to miss the suspected abnormal situation and avoid subsequent repeated inspection.

[0036] It should be understood that when the first inspection plan is adjusted, the inspection plan of the inspection facility corresponding to the suspected abnormal situation is mainly adjusted to perform more comprehensive inspection on the inspection facility corresponding to the suspected abnormal situation. In order to distinguish from other inspection plans, the adjusted inspection plan is recorded as a second inspection plan.

[0037] Step S150, continue to perform the inspection task according to the second inspection plan.

[0038] After adjusting the inspection plan, continue to perform the inspection task according to the second inspection plan obtained by adjustment.

[0039] Exemplarily, if the power inspection scene is used to inspect the power transmission line, when the temperature of a connection part on a tower is abnormally high detected by an infrared thermal imager, the inspection plan can be adjusted to reduce the flight height and use a high-definition camera to take pictures from multiple angles.

[0040] The unmanned aerial vehicle inspection method provided by the application comprises the following steps: obtaining task information to be inspected and historical fault data corresponding to the task information to be inspected; determining a first inspection plan according to the task information to be inspected and the historical fault data; performing an inspection task according to the first inspection plan and obtaining real-time inspection data; adjusting the first inspection plan when a suspected abnormal situation is detected according to the real-time inspection data to obtain a second inspection plan; and continuing to perform the inspection task according to the second inspection plan. In the application, the first inspection plan is formulated based on the task information to be inspected and in combination with the historical fault data to determine a key inspection area and an inspection parameter. Further, the real-time inspection data is analyzed during the performance of the inspection task, the first inspection plan is adjusted when a suspected abnormal situation is detected, and then the task is continued to be performed based on the second inspection plan obtained by adjustment. The inspection plan is dynamically adjusted according to the abnormal detection result detected by the real-time inspection data, so that the inspection work can be more targeted, and subsequent re-inspection can be avoided, thereby improving the efficiency and intelligent level of the unmanned aerial vehicle inspection. In addition, the real-time inspection data is preliminarily detected locally on the unmanned aerial vehicle, so that the data processing amount of the subsequent manual or control end can be reduced, thereby further improving the overall inspection efficiency.

[0041] In an embodiment, after the step S140, the unmanned aerial vehicle inspection method further comprises steps S160, S170 and S180.

[0042] Step S160, obtaining the current task progress.

[0043] In this embodiment, considering that when a large area or a complex facility needs to be inspected, multiple UAVs are usually organized to form a formation to cooperatively perform the inspection task. When the inspection plan of a UAV changes, it may affect the overall inspection efficiency. Therefore, to further improve the inspection efficiency, in this embodiment, the current task progress and the adjusted inspection plan (i.e., the second inspection plan) are shared with other UAVs through communication, so as to further dynamically allocate inspection tasks according to the overall task demand and the current completion situation, form a new inspection plan (i.e., the third inspection plan), and further improve the overall inspection efficiency.

[0044] Here, the current task progress refers to the inspection facilities and / or inspection items that have been completed by the UAV according to the first inspection plan, and of course, it can also include the inspection facilities and / or inspection items to be completed.

[0045] In step S170, the current task progress and the second inspection plan are sent to the target UAV through ad hoc network communication.

[0046] Here, ad hoc network communication is a wireless network technology that does not rely on pre-established fixed infrastructure (such as cellular base stations, Wi-Fi routers, satellite relay stations). The nodes (i.e., each UAV in the UAV group) in the network can dynamically and automatically discover each other and spontaneously form a temporary, multi-hop, decentralized (or weakly centralized) communication network. Through ad hoc network communication, the fundamental limitations of traditional communication methods that rely on fixed base stations in terms of mobility, coverage, deployment cost and robustness can be solved, so that the UAV group can share task progress in real time, receive adjustment instructions, and thus efficiently, flexibly and reliably complete complex cooperative inspection tasks and improve overall inspection efficiency.

[0047] The target UAV can be a lead UAV in the pre-set UAV group, or other UAVs in the pre-set UAV group, which is responsible for dynamically coordinating the inspection plans of each UAV.

[0048] It should be understood that the sending of the current task progress and the second inspection plan can be triggered when the second inspection plan is generated, or it can be periodic.

[0049] When the target UAV receives the current task progress and the second inspection plan sent by the current UAV, it can obtain the current task progress and the current inspection plan of other UAVs; then, it further analyzes whether the current task progress of the current UAV and other UAVs is consistent with the corresponding inspection plan; if not, it can further identify the current situation of each UAV, including but not limited to progress deviation, unexpected situation or resource bottleneck, etc.; and then, according to the identification result, it adjusts the current inspection plan of each UAV.

[0050] Exemplarily, if it is judged that the task of the current UAV is delayed, and at least one of the other UAVs is ahead of schedule, the delayed current UAV can share part of the task, at this time, the second inspection plan of the current UAV can be adjusted to obtain a third inspection plan.

[0051] Upon receiving the third inspection plan fed back by the target UAV, step S180 is performed: according to the third inspection plan, the inspection task is continued to be performed; Upon receiving the third inspection plan fed back by the target UAV, step S150 is performed: according to the second inspection plan, the inspection task is continued to be performed.

[0052] The target UAV, after adjusting the second inspection plan to obtain the third inspection plan, feeds back to the UAV (i.e., the current UAV) that sent the second inspection plan through ad hoc network communication. Correspondingly, the current UAV can receive the third inspection plan fed back by the target UAV through ad hoc network communication.

[0053] Upon receiving the third inspection plan, the inspection task is continued to be performed according to the third inspection plan.

[0054] Upon not receiving the third inspection plan, the inspection task can be performed according to the second inspection plan first.

[0055] In this embodiment, the current task progress and the adjusted inspection plan (i.e., the second inspection plan) are shared in real time through ad hoc network communication, the third inspection plan is dynamically generated and fed back by the target UAV based on global information, and then the inspection task is continued to be performed according to the third inspection plan. Through the above-mentioned multi-UAV cooperative working mechanism, the resources of the entire UAV group can be fully utilized, so that the UAV group can more efficiently cope with dynamic changes, the completion time of the overall inspection task can be shortened, and the overall inspection efficiency can be further improved.

[0056] Figure 2 is a flowchart of the UAV inspection method provided by the present application, as shown in Figure 2 The to-be-inspected task information includes the facility type and the facility location of the to-be-inspected facility, and the above-mentioned step S120 includes step S121, step S122 and step S123.

[0057] Step S121: a first fault prediction model is used to predict the potential fault of the facility according to the facility type and the historical fault data, to obtain a facility fault prediction result.

[0058] Step S122: a second fault prediction model is used to predict the fault area according to the facility location and the historical fault data, to obtain a fault area prediction result.

[0059] Step S123, determining a patrol parameter according to the facility failure prediction result, and determining a key patrol area according to the failure area prediction result.

[0060] The first patrol plan includes the patrol parameter and the key patrol area.

[0061] It should be noted that the execution order of steps S121 and S122 is not sequential.

[0062] In this embodiment, the facility type can include, but is not limited to, photovoltaic panels, towers and oil pipelines, etc.; the facility location includes, but is not limited to, GPS coordinates, regional labels, etc.; the historical failure data includes, but is not limited to, failure areas, failure types, failure levels and failure occurrence times, etc.

[0063] The first failure prediction model and the second failure prediction model can be machine learning models, the first failure prediction model is trained based on the first sample facility failure data and the labeled multi-classification failure type label, and the second failure prediction model is trained based on the second sample facility failure data and the labeled failure area label. The first sample facility failure data and the second sample facility failure data can be the same or different.

[0064] Further, the first failure prediction model can be a LightGBM (Light Gradient Boosting Machine) model, an XGBoost (eXtreme Gradient Boosting) model or a random forest model. Compared with other types of machine learning models, the decision tree model has stronger ability to process mixed data types, can capture complex nonlinear relationships and interactions, and improve the accuracy of the failure area prediction result.

[0065] Further, the second failure prediction model can be a graph neural network or a spatial convolutional neural network. Compared with other types of machine learning models, the graph neural network is better at processing graph structure data and can learn the information propagation and dependence between nodes to identify vulnerable areas, while the spatial convolutional neural network is better at processing spatial data with regular grid structure (such as grid-divided photovoltaic panel images and pipeline segmented heat maps) and capturing local spatial patterns, thereby improving the accuracy of the failure area prediction result.

[0066] Here, the facility failure prediction result can include the failure type and the failure occurrence probability, and the failure area prediction result includes the high failure risk area or the high failure risk location.

[0067] After obtaining the facility failure prediction result and the failure area prediction result, a patrol parameter is determined according to the facility failure prediction result, and a key patrol area is determined according to the failure area prediction result. The patrol parameter is some parameter in a patrol task execution process, including but not limited to: flight height, flight speed, data acquisition parameter. The key patrol area is an area or part that needs to be patrolled.

[0068] In this embodiment, the potential failure of the facility and the failure area are predicted by the failure prediction model, so that a more accurate initial patrol plan (i.e., the first patrol plan) can be made, and the patrol efficiency is improved.

[0069] Further, on the basis of the to-be-patrolled task information and the historical failure data, real-time environment data can also be obtained, so as to determine the first execution plan according to the to-be-patrolled task information, the historical failure data and the real-time environment data. By adding the real-time environment data, the accuracy of the facility failure prediction result and the failure area prediction result can be further improved, so that a more accurate patrol plan can be made, and the patrol efficiency is improved.

[0070] Specifically, the potential failure of the facility is predicted by the first failure prediction model according to the facility type, the historical failure data and the real-time environment data, so as to obtain the facility failure prediction result. Meanwhile, the failure area is predicted by the second failure prediction model according to the facility location, the historical failure data and the real-time environment data, so as to obtain the failure area prediction result. Then, the patrol parameter is determined according to the facility failure prediction result, and the key patrol area is determined according to the failure area prediction result. The types of the first failure prediction model and the second failure prediction model can refer to the above-mentioned embodiments, and the main difference lies in that the training samples of the first failure prediction model and the second failure prediction model need to add sample environment data.

[0071] In an embodiment, step S130 can include step S131 and step S132.

[0072] Step S131, obtaining an initial flight plan.

[0073] Step S132, executing a patrol task according to the first patrol plan and the initial patrol path, and obtaining real-time patrol data and real-time environment data.

[0074] Considering that the current unmanned aerial vehicle inspection has poor environmental adaptability, for example, in complex terrain areas such as mountainous areas and jungles, the signal of the unmanned aerial vehicle is easily blocked and weakened or interrupted, causing the unmanned aerial vehicle to be unable to receive data in real time, so as to obtain obstacle information, and finally it is difficult to avoid obstacles or lose control. In addition, when encountering strong winds, heavy rain, thick fog and other bad weather, the flight stability of the unmanned aerial vehicle will also be seriously affected, and it may even be unable to normally take off to perform the inspection task. Therefore, in the embodiment, during the inspection process, real-time environmental data is obtained to adjust the flight plan in real time, so as to realize autonomous obstacle avoidance flight and maintain flight stability, thereby improving the safety of the unmanned aerial vehicle inspection.

[0075] Here, the initial flight plan includes an initial flight path and initial flight parameters. The initial flight path can be determined according to the facility location, the task starting point and the task ending point, and the initial flight parameters can be determined according to the facility type and the inspection purpose. The initial flight parameters can include but are not limited to flight attitude parameters and flight power parameters, wherein the flight attitude parameters include but are not limited to pitch angle, roll angle, yaw angle and angular velocity, and the flight power parameters include but are not limited to total thrust, motor differential thrust and motor speed.

[0076] When performing the inspection task, the first inspection plan and the initial inspection path are executed. During the execution of the inspection task, real-time environmental data is obtained while obtaining real-time inspection data. The real-time environmental data includes but is not limited to three-dimensional point cloud data, obstacle distance, barometric altitude and weather data. The three-dimensional point cloud data can be obtained by a laser radar, the obstacle distance can be obtained by an ultrasonic sensor, the barometric altitude can be obtained by a barometric altimeter, and the weather data can include but is not limited to wind speed, wind direction, temperature, humidity and air pressure. Correspondingly, the wind speed and the wind direction can be obtained by a wind speed sensor, and the temperature, the humidity and the air pressure can be obtained by a temperature sensor, a humidity sensor and an air pressure sensor, respectively.

[0077] Further, after the above step S130, it further includes: Step S190, adjusting the initial flight plan according to the real-time environmental data to obtain an adjusted flight plan.

[0078] The adjusted flight plan includes an adjusted flight path and / or an adjusted flight parameter, that is, either or both of the flight path and the flight parameter can be adjusted according to the real-time environment data. The adjusted flight path can be obtained in the following manner: first, a real-time three-dimensional model of the flight environment is constructed according to the three-dimensional point cloud data and the obstacle distance; then, real-time terrain and obstacle information is identified according to the real-time three-dimensional model and the barometric altitude; and then, the initial flight path is adjusted according to the real-time terrain and obstacle information to obtain the adjusted flight path. The adjusted flight parameter can be obtained in the following manner: the initial flight parameter is adjusted according to the meteorological data to obtain the adjusted flight parameter. The specific execution process can be referred to the following embodiments.

[0079] At this time, the step S150 includes: The step S151 continues to execute the inspection task according to the second inspection plan and the adjusted flight plan.

[0080] After the flight plan is adjusted, the inspection task can be continued according to the second inspection plan and the adjusted flight plan.

[0081] It should be understood that if the first inspection plan is not adjusted, only the flight plan is adjusted, then the inspection task is continued according to the first inspection plan and the adjusted flight plan.

[0082] In this embodiment, the real-time flight plan is adjusted by obtaining real-time environment data during the inspection process of the unmanned aerial vehicle, so as to improve the environmental adaptability of the unmanned aerial vehicle inspection in complex environment, realize autonomous obstacle avoidance flight and maintain flight stability, thereby improving the safety of the unmanned aerial vehicle inspection. At the same time, compared with relying on manual remote intervention, the flight plan is adjusted in real time on the unmanned aerial vehicle in this embodiment, so that the problem can be solved timely and effectively when a sudden situation is encountered in a complex environment.

[0083] In an embodiment, the initial flight plan includes an initial flight path and an initial flight parameter, and the real-time environment data includes three-dimensional point cloud data, obstacle distance, barometric altitude and meteorological data. The step S190 includes the steps S191, S192, S193 and S194.

[0084] The step S191 constructs a real-time three-dimensional model of the flight environment according to the three-dimensional point cloud data and the obstacle distance.

[0085] Here, the three-dimensional point cloud data can be obtained by a laser radar, and the obstacle distance can be obtained by an ultrasonic sensor.

[0086] The three-dimensional point cloud data is filtered by a voxel grid to reduce the data amount. Then, the obstacle distance value is mapped to the point cloud coordinate system to supplement the near-ground blind area data. Further, the multiple frames of three-dimensional point cloud data filtered by the voxel grid are registered based on an ICP (Iterative Closest Point) algorithm to generate a real-time three-dimensional model covering the flight area.

[0087] Further, before the real-time three-dimensional model is constructed, the three-dimensional point cloud data can be preprocessed, and then the real-time three-dimensional model of the flight environment is constructed based on the preprocessed three-dimensional point cloud data and the obstacle distance. The preprocessing of the three-dimensional point cloud data includes but is not limited to denoising and filtering. Through denoising and filtering, the quality of the original three-dimensional point cloud data can be improved to facilitate subsequent processing.

[0088] In step S192, the real-time terrain and obstacle information are identified based on the real-time three-dimensional model and the barometric altitude.

[0089] Here, the barometric altitude can be obtained by a barometric altimeter.

[0090] The real-time three-dimensional model is segmented by a RANSAC (Random Sample Consensus) plane fitting algorithm to obtain ground point cloud and non-ground point cloud, and then the relative elevation corresponding to the ground point cloud is obtained. Combined with the barometric altitude, the absolute elevation is calibrated, the terrain slope angle is calculated according to the absolute elevation, and then the real-time terrain is determined according to the calculation result. For example, the part with a terrain slope angle > 30° is marked as a steep slope, the part with a terrain slope angle < -25° is marked as a cliff, and the other part is marked as a flat ground. The non-ground point cloud is aggregated into an independent obstacle by using a Euclidean clustering algorithm, and the boundary box position and size of the obstacle (i.e. obstacle information) are output.

[0091] In step S193, the initial flight path is adjusted based on the real-time terrain and the obstacle information to obtain an adjusted flight path.

[0092] The real-time terrain and the obstacle information are input into a preset path planning algorithm to adjust the initial flight path by using the path planning algorithm to obtain an adjusted flight path.

[0093] The preset path planning algorithm can be an A* search algorithm (commonly known as A-star algorithm), which is particularly suitable for dynamic obstacle avoidance and complex terrain navigation scenarios, and has more significant advantages in unmanned aerial vehicle path planning compared with other path planning algorithms.

[0094] In step S194, the initial flight parameters are adjusted based on the weather data to obtain adjusted flight parameters.

[0095] The adjusted flight plan comprises the adjusted flight path and the adjusted flight parameters.

[0096] Here, the meteorological data can include, but is not limited to, wind speed, wind direction, temperature, humidity and air pressure, and correspondingly, the wind speed and wind direction can be obtained by a wind speed sensor, and the temperature, humidity and air pressure can be obtained by a temperature sensor, a humidity sensor and an air pressure sensor respectively.

[0097] The flight parameters include, but are not limited to, flight attitude parameters and flight power parameters, wherein the flight attitude parameters include, but are not limited to, pitch angle, roll angle, yaw angle and angular velocity, and the flight power parameters include, but are not limited to, total thrust, motor differential thrust and motor speed.

[0098] In the adjustment, the adjustment can be made according to a preset adjustment rule, which can be set according to actual needs and is not limited here.

[0099] For example, when it is determined that it is strong wind and bad weather according to the wind speed, the flight attitude parameters and the flight power parameters can be adjusted according to the wind speed and the wind direction to maintain flight stability.

[0100] In this embodiment, the initial flight path and the initial flight parameters are adjusted according to real-time environmental data to improve the environmental adaptability of the unmanned aerial vehicle inspection in complex environments, realize autonomous obstacle avoidance flight and maintain flight stability, thereby improving the safety of the unmanned aerial vehicle inspection.

[0101] Based on any of the above embodiments, after step S130, the unmanned aerial vehicle inspection method further comprises: When a suspected abnormal situation is detected according to the real-time inspection data, the real-time inspection data corresponding to the suspected abnormal situation is sent to the control end for analysis and processing by the control end.

[0102] In the prior art, all detected inspection data is usually sent to the ground control station and / or the cloud server when the entire inspection task is completed, and the ground control station and / or the cloud server and / or artificial analysis of the massive inspection data is performed, which has the problems of low efficiency and untimely abnormality processing.

[0103] Therefore, in this embodiment, when a suspected abnormality is detected based on real-time inspection data, the real-time inspection data corresponding to the suspected abnormality is sent to the control terminal, which can be a ground control station or a cloud server, for analysis and processing by the control terminal. Through the above method, when a suspected abnormality is discovered, the control terminal can promptly analyze and process the relevant inspection data to provide abnormality diagnosis results and suggestions, thereby promptly repairing the abnormal facilities or taking other timely measures to deal with the abnormality. In addition, by performing preliminary processing and screening of the real-time inspection data locally on the drone, removing redundant information, and then only sending the real-time inspection data corresponding to the suspected abnormality to the control terminal, the data processing volume of the control terminal can be greatly reduced, and overall efficiency can be improved.

[0104] Furthermore, when a suspected abnormality is detected based on the real-time inspection data, inspection data corresponding to the suspected abnormality can be further obtained according to the second inspection plan and recorded as abnormal inspection data. The real-time inspection data and abnormal inspection data corresponding to the suspected abnormality are then sent to the control end. This method can increase the amount of inspection data related to the suspected abnormality, facilitating a more comprehensive and accurate analysis by the control end.

[0105] For example, during power line inspections, if an infrared thermal imager captures real-time thermal images and detects an abnormally high temperature in a certain part of the line, the inspection plan is adjusted, and the flight position is further adjusted. A high-definition camera captures the area from multiple angles to obtain images of the abnormal part, increasing the frequency of inspections. The real-time thermal images and images of the abnormal part are then sent to the control terminal for analysis and processing.

[0106] The drone inspection device provided by the present invention is described below. The drone inspection device described below and the drone inspection method described above can be referenced to each other.

[0107] Figure 3 This is a schematic diagram of the structure of the drone inspection device provided by the present invention. Figure 3 As shown, the apparatus includes a first acquisition module 310, a determination module 320, a second acquisition module 330, an adjustment module 340 and an execution module 350; wherein: The first acquisition module 310 is used to obtain information of tasks to be inspected and their corresponding historical fault data; A determination module 320 is configured to determine a first inspection plan based on the to-be-inspected task information and the historical fault data; A second acquisition module 330 is configured to execute an inspection task according to the first inspection plan and obtain real-time inspection data; An adjustment module 340 is configured to adjust the first inspection plan to obtain a second inspection plan when a suspected abnormality is detected according to the real-time inspection data; The execution module 350 is used to continue executing the inspection task according to the second inspection plan.

[0108] The drone inspection device provided by the present invention obtains information about tasks to be inspected and their corresponding historical fault data; then, based on the information about tasks to be inspected and the historical fault data, determines a first inspection plan; then, according to the first inspection plan, executes the inspection task and obtains real-time inspection data; when a suspected abnormality is detected based on the real-time inspection data, adjusts the first inspection plan to obtain a second inspection plan; and continues to execute the inspection task based on the second inspection plan. In the present invention, based on the information about tasks to be inspected and combined with historical fault data, a first inspection plan is formulated to determine key inspection areas and inspection parameters. Furthermore, during the execution of the inspection task, the real-time inspection data is analyzed. When a suspected abnormality is detected, the first inspection plan is adjusted, and then, based on the adjusted second inspection plan, the task is continued. By dynamically adjusting the inspection plan based on the abnormality detection results detected by the real-time inspection data, the inspection work can be carried out more specifically, avoiding subsequent re-inspections, thereby improving the efficiency and intelligence level of drone inspections. In addition, by performing preliminary detection of real-time inspection data locally on the drone, the amount of subsequent data processing by manual or control terminals can be reduced, thereby further improving the overall inspection efficiency.

[0109] It should be noted here that the above-mentioned drone inspection device provided in the embodiment of the present invention can implement all the method steps implemented in the above-mentioned drone inspection method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0110] Figure 4 The following is an example of a schematic diagram of the physical structure of a drone, such as Figure 4 As shown, the drone may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the drone inspection method, which includes: Obtain information on pending inspection tasks and their corresponding historical fault data; Determining a first inspection plan based on the to-be-inspected task information and the historical fault data; According to the first inspection plan, an inspection task is performed, and real-time inspection data is acquired; When a suspected abnormal situation is detected according to the real-time inspection data, the first inspection plan is adjusted to obtain a second inspection plan; According to the second inspection plan, the inspection task is continuously performed.

[0111] In addition, the logic instructions in the memory 430 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0112] Further, the unmanned aerial vehicle can further include: an unmanned aerial vehicle body, a sensor, a communication module, and an energy module.

[0113] The unmanned aerial vehicle body is designed as a multi-rotor unmanned aerial vehicle with strong wind resistance and long endurance. The fuselage is equipped with a plurality of high-precision sensors, including a laser radar, an ultrasonic sensor, an air pressure altimeter, etc., for real-time sensing of surrounding environment information. At the same time, a high-resolution visible light camera, an infrared thermal imager, and a sensor with specific detection function (such as a corona detector in power inspection) are carried.

[0114] The communication module integrates multiple communication modes, including but not limited to: 4G / 5G communication, satellite communication, and ad hoc network communication. Among them, 4G / 5G communication is used for regular data transmission; satellite communication is used as a backup communication means to ensure stable communication in areas with poor signal; at the same time, ad hoc network technology is used to enable multiple unmanned aerial vehicles to communicate with each other and work cooperatively.

[0115] The energy module uses high-efficiency batteries and combines solar charging panels to supplement the battery with solar energy during flight, extending the endurance of the unmanned aerial vehicle.

[0116] In another aspect, the present application also provides a computer program product comprising a computer program, which can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to perform the UAV inspection method provided by the above-mentioned methods, which comprises: obtaining task information to be inspected and corresponding historical fault data; determining a first inspection plan according to the task information to be inspected and the historical fault data; performing an inspection task according to the first inspection plan and obtaining real-time inspection data; adjusting the first inspection plan to obtain a second inspection plan when a suspected abnormal situation is detected according to the real-time inspection data; continuing to perform the inspection task according to the second inspection plan.

[0117] In another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, and the computer program can be executed by a processor to implement the UAV inspection method provided by the above-mentioned methods, which comprises: obtaining task information to be inspected and corresponding historical fault data; determining a first inspection plan according to the task information to be inspected and the historical fault data; performing an inspection task according to the first inspection plan and obtaining real-time inspection data; adjusting the first inspection plan to obtain a second inspection plan when a suspected abnormal situation is detected according to the real-time inspection data; continuing to perform the inspection task according to the second inspection plan.

[0118] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement it without creative labor.

[0119] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0120] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A drone inspection method, characterized in that: include: Obtain information on pending inspection tasks and their corresponding historical fault data; Determining a first inspection plan based on the to-be-inspected task information and the historical fault data; According to the first inspection plan, perform inspection tasks and obtain real-time inspection data; When a suspected abnormality is detected according to the real-time inspection data, the first inspection plan is adjusted to obtain a second inspection plan; Continue to perform the inspection task according to the second inspection plan.

2. The drone inspection method according to claim 1, characterized in that: When a suspected abnormality is detected according to the real-time inspection data, the first inspection plan is adjusted to obtain a second inspection plan, further comprising: Get the current task progress; Sending the current task progress and the second inspection plan to the target UAV via ad hoc network communication; Upon receiving the third inspection plan fed back by the target UAV, continue to perform the inspection task according to the third inspection plan; When the third inspection plan fed back by the target UAV is not received, the inspection task is continued according to the second inspection plan.

3. The drone inspection method according to claim 1, characterized in that: The task information to be inspected includes the facility type and facility location of the facility to be inspected. The determining of the first inspection plan based on the task information to be inspected and the historical fault data includes: Predicting potential failures of a facility based on the facility type and the historical failure data using a first failure prediction model to obtain a facility failure prediction result; Using a second fault prediction model, predicting a fault area based on the facility location and the historical fault data to obtain a fault area prediction result; Determine inspection parameters based on the facility fault prediction results, and determine key inspection areas based on the fault area prediction results; The first inspection plan includes the inspection parameters and the key inspection areas.

4. The drone inspection method according to claim 1, characterized in that: The performing of inspection tasks according to the first inspection plan and obtaining real-time inspection data includes: Obtain an initial flight plan; Performing inspection tasks according to the first inspection plan and the initial inspection path, and acquiring real-time inspection data and real-time environmental data; After executing the inspection task according to the first inspection plan and obtaining real-time inspection data, the method further includes: Adjusting the initial flight plan according to the real-time environmental data to obtain an adjusted flight plan; Continuing to perform the inspection task according to the second inspection plan includes: Continue to perform the inspection mission according to the second inspection plan and the adjusted flight plan.

5. The drone inspection method according to claim 4, characterized in that: The initial flight plan includes an initial flight path and initial flight parameters, and the real-time environmental data includes three-dimensional point cloud data, obstacle distance, pressure altitude and meteorological data; The step of adjusting the initial flight plan according to the real-time environmental data to obtain an adjusted flight plan includes: constructing a real-time three-dimensional model of the flight environment based on the three-dimensional point cloud data and the obstacle distance; Identify and obtain real-time terrain and obstacle information based on the real-time three-dimensional model and the air pressure altitude; Adjusting the initial flight path according to the real-time terrain and the obstacle information to obtain an adjusted flight path; adjusting the initial flight parameters according to the meteorological data to obtain adjusted flight parameters; The adjusted flight plan includes the adjusted flight path and the adjusted flight parameters.

6. The drone inspection method according to any one of claims 1 to 5, characterized in that: After obtaining the real-time inspection data, the method further includes: When a suspected abnormal situation is detected based on the real-time inspection data, the real-time inspection data corresponding to the suspected abnormal situation is sent to the control end for analysis and processing by the control end.

7. A drone inspection device, characterized in that: include: The first acquisition module is used to obtain the information of the inspection task and its corresponding historical fault data; A determination module, configured to determine a first inspection plan based on the to-be-inspected task information and the historical fault data; A second acquisition module is used to execute the inspection task according to the first inspection plan and obtain real-time inspection data; an adjustment module, configured to adjust the first inspection plan to obtain a second inspection plan when a suspected abnormality is detected according to the real-time inspection data; An execution module is used to continue executing the inspection task according to the second inspection plan.

8. A drone comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the drone inspection method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the drone inspection method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the drone inspection method according to any one of claims 1 to 6 is implemented.

Images