Unmanned aerial vehicle autonomous inspection method and system and storage medium

Abstract

The invention discloses an unmanned aerial vehicle autonomous inspection method and system and a storage medium. The method is suitable for an inspection unmanned aerial vehicle of the unmanned aerial vehicle autonomous inspection system. The method comprises the following steps: acquiring an initial inspection instruction and an initial shooting parameter transmitted by a control center in the unmanned aerial vehicle autonomous inspection system; acquiring a shot image of the to-be-inspected target point according to the comprehensive evaluation value of the to-be-inspected target point, the initial inspection instruction and the current shooting parameter; the current shooting parameter during initial acquisition is an initial shooting parameter; inputting the shot image into a preset target recognition model for accuracy recognition to obtain a target accuracy; judging whether the target accuracy is greater than a preset threshold; if yes, the shot image is transmitted to the control center; and if not, adjusting the current shooting parameter according to the target accuracy rate until the target accuracy rate is greater than a preset threshold value, obtaining a new shot image, and transmitting the new shot image to the control center. According to the invention, the accuracy of unmanned aerial vehicle inspection shooting images is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of substation inspection, and in particular to an autonomous inspection method, system and storage medium of an unmanned aerial vehicle. Background Art

[0002] With the continuous development of science and technology, drone technology has made significant progress in various fields, including wireless communication support, infrastructure inspection, monitoring and surveillance, and power equipment inspection. Compared with manual labor, drone inspection technology reduces the interference of human activities on the environment and reduces the consumption of natural resources for inspections. However, the effectiveness and quality of drone inspections are currently affected by various factors. During substation inspections, environmental factors such as complex airflow can affect the position and attitude accuracy of drones, resulting in the camera being unable to accurately aim at the target equipment for photography. Traditional drone inspection methods use fixed routes, fixed shooting positions, and fixed shooting angles. They are unable to cope with the impact of positioning errors caused by environmental factors. As a result, the shooting position during drone inspections is easily affected by external environmental disturbances, making it difficult to obtain high-quality inspection photos, resulting in low drone inspection accuracy. Summary of the Invention

[0003] The embodiments of the present invention provide a method, system and storage medium for autonomous inspection of drones, which can effectively solve the problem that the existing drone inspection method has a fixed route, fixed shooting position and fixed shooting angle, and is unable to cope with the influence of positioning errors caused by environmental factors, resulting in the shooting position of the drone during the inspection process being easily affected by external environmental disturbances, making it difficult to obtain high-quality inspection photos, resulting in low accuracy of drone inspections.

[0004] An embodiment of the present invention provides an autonomous inspection method for a drone, which is applicable to an inspection drone of a drone autonomous inspection system; the drone autonomous inspection system also includes a control center; the autonomous inspection method for a drone includes:

[0005] Obtaining the geometric distance between the current position and several target points to be inspected, the regional hazard level of each target point to be inspected, the priority of the target device, the initial inspection instruction transmitted by the control center, and the initial shooting parameters;

[0006] A comprehensive evaluation value of each target point to be inspected is obtained by weighted calculation based on the geometric distance, the regional danger level, and the target device priority;

[0007] According to the comprehensive evaluation value, the initial inspection instruction and the current shooting parameters, the image of the target point to be inspected is collected; wherein the current shooting parameters at the time of initial collection are the initial shooting parameters;

[0008] Inputting the captured image into a preset target recognition model for accuracy recognition to obtain the target accuracy of the captured image;

[0009] According to the target accuracy rate and a preset threshold, determining whether the target accuracy rate is greater than the preset threshold rate;

[0010] If yes, transmitting the captured image to the control center so that the control center stores the captured image;

[0011] If not, adjust the shooting distance and deflection angle according to the target accuracy, and adjust the current shooting parameters according to the shooting distance and deflection angle until the target accuracy is greater than the preset threshold, obtain a new shooting image, and transmit the new shooting image to the control center so that the control center stores the new shooting image.

[0012] Furthermore, the control center is used to generate an initial inspection instruction and transmit the initial inspection instruction to the inspection drone;

[0013] The generation of the initial inspection instruction includes:

[0014] Obtain route data, waypoint data, target point location data, and target equipment information of the target point to be inspected;

[0015] Performing path planning based on the route data, the waypoint data, the target point location data, and the target device information to generate an initial inspection route;

[0016] An initial inspection instruction is generated according to the initial inspection route.

[0017] Furthermore, according to the comprehensive evaluation value, the initial inspection instruction and the current shooting parameters, a photographic image of the target point to be inspected is collected, including:

[0018] According to the initial inspection instruction, parsing the initial inspection route in the initial inspection instruction;

[0019] Rearranging the target points to be inspected on the initial inspection route according to the order of the comprehensive evaluation values ​​from large to small to generate a new inspection route;

[0020] Arrive at the corresponding shooting location according to the new inspection route, and adjust the posture, gimbal angle, and camera shooting parameters according to the current shooting parameters at the corresponding shooting location;

[0021] According to the adjusted posture, pan / tilt angle and camera shooting parameters, the camera collects images of the target points to be inspected.

[0022] Furthermore, the target recognition model is deployed on the inspection drone; the training of the target recognition model includes:

[0023] Obtain historical inspection images and target recognition results;

[0024] Marking the target edge, target position and target angle of the historical inspection image to obtain a target historical image;

[0025] Inputting the target historical image into the target recognition model to be trained, training according to the current model parameters, and obtaining a predicted recognition result; wherein the current model parameters at the initial time are the initial model parameters;

[0026] Calculating the recognition accuracy rate based on the predicted recognition results and the historical recognition results;

[0027] When the recognition accuracy is determined to have converged, the trained target recognition model is obtained;

[0028] When it is determined that the recognition accuracy has not converged, the current model parameters are updated according to the recognition accuracy, and the updated current model parameters are used as the current model parameters for the next training.

[0029] Furthermore, adjusting the shooting distance and the deflection angle according to the target accuracy, and adjusting the current shooting parameters according to the shooting distance and the deflection angle, includes:

[0030] Get the current location;

[0031] Determining, according to the target accuracy, a first size, a first position, and a first angle of the corresponding target point to be inspected in the captured image;

[0032] Calculating a shooting distance between a target point to be inspected and a current position according to the first size and the first position; and adjusting the shooting distance according to the first size and the first position;

[0033] Calculating a deflection angle between the target point to be inspected and the current position based on the first angle and the first position; and adjusting the deflection angle based on the first angle and the first position;

[0034] According to the adjusted shooting distance and the adjusted deflection angle, adjust the current shooting parameters' own posture, gimbal angle, and camera shooting parameters.

[0035] Furthermore, the control center is used to perform path planning based on the route data, the waypoint data, the target point location data and the target device information to generate an initial inspection route;

[0036] The route data includes: route number and route type;

[0037] The waypoint data includes: the longitude and latitude of the waypoint and the altitude of the waypoint;

[0038] The target point location data includes: the target point latitude and longitude and the target point height of the target point to be inspected;

[0039] The target device information includes: the target device priority and device height of the target point to be inspected;

[0040] Performing path planning based on the route data, the waypoint data, the target point location data, and the target device information to generate an initial inspection route includes:

[0041] Matching the corresponding route type and waypoint data according to the route number; and determining the spatial correspondence between the longitude and latitude of the waypoint and the longitude and latitude of the target point according to the matched waypoint data;

[0042] According to the spatial correspondence, based on the waypoint altitude, the target point altitude and the device altitude, a reference flight altitude is calculated to meet the flight requirements;

[0043] Generate an initial target route using the matched waypoint data as path nodes and the reference flight altitude; and map the target point location data to the initial target route to generate an inspection path segment;

[0044] The inspection path segments are weighted according to the priority of the target device, so that the positions of the inspection target points are sorted in descending order of the priority of the target device, and the initial inspection route is obtained.

[0045] As an improvement to the above solution, another embodiment of the present invention provides a drone autonomous inspection system, including an inspection drone and a control center;

[0046] The inspection drone is used to obtain the geometric distance between the current position and several target points to be inspected, the regional hazard level of each target point to be inspected, the target device priority, the initial inspection instruction transmitted by the control center, and the initial shooting parameters; based on the geometric distance, the regional hazard level, and the target device priority, a weighted calculation is performed to obtain a comprehensive evaluation value of each target point to be inspected; based on the comprehensive evaluation value, the initial inspection instruction, and the current shooting parameters, a photographic image of the target point to be inspected is collected; wherein the current shooting parameters at the time of initial collection are the initial shooting parameters; the photographic image is input into a preset target recognition model for accuracy recognition to obtain a target accuracy of the photographic image; based on the target accuracy and a preset threshold, it is determined whether the target accuracy is greater than a preset threshold; if so, the photographic image is transmitted to the control center so that the control center stores the photographic image; if not, the shooting distance and deflection angle are adjusted according to the target accuracy, and the current shooting parameters are adjusted according to the shooting distance and deflection angle until the target accuracy is greater than the preset threshold, thereby obtaining a new photographic image, and the new photographic image is transmitted to the control center so that the control center stores the new photographic image;

[0047] The control center is used to store the captured images transmitted by the inspection drone and new captured images.

[0048] Furthermore, the control center is further configured to generate an initial inspection instruction and transmit the initial inspection instruction to the inspection drone;

[0049] The generation of the initial inspection instruction includes:

[0050] Obtain route data, waypoint data, target point location data, and target equipment information of the target point to be inspected;

[0051] Performing path planning based on the route data, the waypoint data, the target point location data, and the target device information to generate an initial inspection route;

[0052] An initial inspection instruction is generated according to the initial inspection route.

[0053] Furthermore, the inspection drone is further configured to collect images of the target point to be inspected based on the comprehensive evaluation value, the initial inspection instruction, and current shooting parameters;

[0054] According to the initial inspection instruction and the current shooting parameters, an image of the target point to be inspected is collected, including:

[0055] According to the initial inspection instruction, parsing the initial inspection route in the initial inspection instruction;

[0056] Rearranging the target points to be inspected on the initial inspection route according to the order of the comprehensive evaluation values ​​from large to small to generate a new inspection route;

[0057] Arrive at the corresponding shooting location according to the new inspection route, and adjust the posture, gimbal angle, and camera shooting parameters according to the current shooting parameters at the corresponding shooting location;

[0058] According to the adjusted posture, pan / tilt angle and camera shooting parameters, the camera collects images of the target points to be inspected.

[0059] Another embodiment of the present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the drone autonomous inspection method described in the above embodiment.

[0060] By implementing the present invention, at least the following beneficial effects are achieved:

[0061] The present invention provides an autonomous inspection method, system and storage medium of an unmanned aerial vehicle (UAV) and the method is applicable to an inspection UAV of an autonomous inspection system of an UAV; the UAV autonomous inspection system also includes a control center; the UAV autonomous inspection method comprises: obtaining the geometric distance between the current position and a number of target points to be inspected, the regional hazard level of each target point to be inspected, the priority of the target device, the initial inspection instruction transmitted by the control center and the initial shooting parameters; according to the geometric distance, the regional hazard level and the priority of the target device, obtaining the comprehensive evaluation value of each target point to be inspected by weighted calculation; according to the comprehensive evaluation value, the initial inspection instruction and the current shooting parameters, collecting the shooting image of the target point to be inspected; wherein, the initial shooting parameter is used. The current shooting parameters at the time of collection are the initial shooting parameters; the captured image is input into a preset target recognition model for accuracy recognition to obtain the target accuracy of the captured image; according to the target accuracy and a preset threshold, it is determined whether the target accuracy is greater than the preset threshold; if so, the captured image is transmitted to the control center so that the control center stores the captured image; if not, the shooting distance and the deflection angle are adjusted according to the target accuracy, and the current shooting parameters are adjusted according to the shooting distance and the deflection angle until the target accuracy is greater than the preset threshold, a new captured image is obtained, and the new captured image is transmitted to the control center so that the control center stores the new captured image. The drone can automatically collect and shoot images based on the initial inspection instructions of the control center, the current shooting parameters and the comprehensive evaluation values ​​of each target point to be inspected, reducing human intervention; the accuracy of the captured images is recognized through the preset target recognition model, and the current shooting parameters are adjusted according to the target accuracy so that the target accuracy is greater than the preset threshold, ensuring that the collected images have a high recognition accuracy. Even if the target accuracy does not reach the preset threshold due to the influence of the external environment, the drone can dynamically adjust its own shooting distance and deflection angle, and then adjust the current shooting parameters according to the adjusted shooting distance and deflection angle to obtain high-quality photos, thereby improving the accuracy of images taken during inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a method for autonomous inspection by a drone provided by one embodiment of the present invention;

[0063] Figure 2 This is a schematic diagram of communication between an inspection drone and a control center provided by an embodiment of the present invention;

[0064] Figure 3 The figure is a schematic structural diagram of an autonomous inspection system for unmanned aerial vehicles provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0066] See also Figure 1 To address the difficulty in obtaining accurate inspection photos during drone inspections, an embodiment of the present invention provides a flowchart of a drone autonomous inspection method, which is applicable to an inspection drone in a drone autonomous inspection system; the drone autonomous inspection system also includes a control center; the drone autonomous inspection method includes:

[0067] S1. Obtaining the geometric distance between the current position and several target points to be inspected, the regional hazard level of each target point to be inspected, the priority of the target device, the initial inspection instruction transmitted by the control center, and the initial shooting parameters;

[0068] Specifically, the control center is used to generate an initial inspection instruction and transmit the initial inspection instruction to the inspection drone;

[0069] The generation of the initial inspection instruction includes:

[0070] Obtain route data, waypoint data, target point location data, and target equipment information of the target point to be inspected;

[0071] Performing path planning based on the route data, the waypoint data, the target point location data, and the target device information to generate an initial inspection route;

[0072] An initial inspection instruction is generated according to the initial inspection route.

[0073] In a preferred embodiment of the present invention, the geometric distance between the current location and the multiple inspection targets is the straight-line distance between the drone's current location and the multiple inspection targets. The regional hazard level of each inspection target represents the regional hazard level of the area in which the inspection target is located. For example, if the inspection target is located in a stormy environment, the regional hazard level will be higher than if the area is clear and flat. The target device priority represents the priority of the target device at the inspection target node. For example, if a transformer suddenly emits smoke, the target device will have a higher priority than a transformer that is not emitting smoke. The initial inspection command contains key information such as the drone's takeoff time, takeoff location, flight speed, flight altitude, waypoint sequence, and target point photography requirements. This information ensures that the drone can perform inspections according to the predetermined route and parameters. After generating the initial inspection command, the control center transmits it to the inspection drone via wireless communication. Upon receiving the command, the drone will execute the inspection task according to the command. Through automated path planning and command generation, the control center can quickly plan the optimal inspection route for the drone, reducing the time and cost of manual planning.

[0074] Specifically, the control center is used to perform path planning based on the route data, the waypoint data, the target point location data and the target device information to generate an initial inspection route;

[0075] The route data includes: route number and route type;

[0076] The waypoint data includes: the longitude and latitude of the waypoint and the altitude of the waypoint;

[0077] The target point location data includes: the target point latitude and longitude and the target point height of the target point to be inspected;

[0078] The target device information includes: the target device priority and device height of the target point to be inspected;

[0079] Performing path planning based on the route data, the waypoint data, the target point location data, and the target device information to generate an initial inspection route includes:

[0080] Matching the corresponding route type and waypoint data according to the route number; and determining the spatial correspondence between the longitude and latitude of the waypoint and the longitude and latitude of the target point according to the matched waypoint data;

[0081] According to the spatial correspondence, based on the waypoint altitude, the target point altitude and the device altitude, a reference flight altitude is calculated to meet the flight requirements;

[0082] Generate an initial target route using the matched waypoint data as path nodes and the reference flight altitude; and map the target point location data to the initial target route to generate an inspection path segment;

[0083] The inspection path segments are weighted according to the priority of the target device, so that the positions of the inspection target points are sorted in descending order of the priority of the target device, and the initial inspection route is obtained.

[0084] In a preferred embodiment of the present invention, the control center first matches the route type and waypoint data based on the route number. The route number is a unique identifier that allows for quick location of a specific route and its associated waypoint data. The waypoint data contains the waypoint's latitude, longitude, and altitude information, which serves as the basis for constructing the inspection route. After matching the waypoint data, the control center further determines the spatial correspondence between the waypoint's longitude and latitude and the target point's longitude and latitude. Utilizing advanced Geographic Information System (GIS) technology, spatial algorithms are used to accurately calculate the relative positional relationship between the waypoint and target point. Next, based on this spatial correspondence, the control center calculates a reference flight altitude based on the waypoint altitude, target altitude, and equipment altitude. This process comprehensively considers factors such as the drone's flight performance, safety margin, and the target equipment's altitude restrictions. This calculation ensures that the drone will neither collide with the ground or obstacles during flight nor affect the normal operation of the target equipment due to flying too low. After determining the reference flight altitude, the initial target route is generated using the matched waypoint data as path nodes and the reference flight altitude. The initial target route is a preliminary plan for the drone's flight path from its takeoff point to each waypoint. The control center then maps the target point location data onto the initial target route, generating inspection path segments. This ensures the drone can inspect each target point in the predetermined order and path. Finally, the inspection path segments are weighted based on the target device priority. Target device priority reflects the importance and urgency of the target point to be inspected. Through weighted assignment, the control center ensures that the drone prioritizes inspections of higher-priority locations among the inspection targets. The inspection path segments are sorted from high to low according to target device priority, ultimately resulting in the initial inspection route. This weighted assignment of target device priority further enhances the targeted nature of inspections, ensuring that important equipment receives priority inspection and reducing the risk of missed and false detections. The system can process different types of route data and target device information to meet inspection needs in diverse scenarios.

[0085] In another preferred embodiment of the present invention, for a certain large outdoor substation, the station contains multiple sets of transformers, high-voltage switchgear, busbars, insulators, lightning rods and other equipment, which are distributed in different areas. Preliminary marking of areas where hidden dangers may exist; using high-precision three-dimensional laser and visible light to build a three-dimensional model of the substation and mark the location, size, longitude and latitude of each device, as well as geographic information such as roads and obstacles within the station. The improved A* algorithm is used as the main route planning algorithm. First, a comprehensive evaluation function is defined: f = w1*D1+w2*D2+w3*P, where w i The weights are preset and can be adjusted dynamically based on actual needs. D1 represents the geometric distance to the target point, D2 represents the danger level of the area near the route, that is, the regional danger level of each target point to be inspected, and P represents the priority of the target equipment. Using the drone's initial position as the starting point and the locations of all equipment to be inspected within the substation as the target points, an improved A* algorithm is used to calculate an initial target route from the starting point to each target node. During the inspection process, if a new emergency anomaly is detected in a piece of equipment, such as sudden smoke from a transformer or an unusual sound from a switchgear, the current route is immediately interrupted and the aircraft is flown directly to the anomalous equipment at maximum speed, simultaneously sending an alert to the monitoring center. After completing the emergency anomaly handling (such as capturing images and recording data), the improved A* algorithm is re-applied to plan the subsequent route based on the remaining uninspected equipment and its current location. Assuming the drone takes off from the substation's charging platform, it first heads to a group of transformers with high oil temperatures according to the initial target route calculated by the comprehensive cost function. After completing the inspection, it proceeds to the switchgear area. When the drone discovered that the indicator light of a switch cabinet was flashing abnormally during the journey, it immediately interrupted its original plan and headed straight for the switch cabinet. After recording the detailed information, it replanned the subsequent route based on the remaining uninspected equipment and its own power and signal conditions, and went to inspect the busbar, insulators and other areas before returning to the charging platform.

[0086] S2. Obtain a comprehensive evaluation value of each target point to be inspected by performing weighted calculation based on the geometric distance, the regional danger level, and the target device priority;

[0087] In a preferred embodiment of the present invention, the comprehensive evaluation value represents the comprehensive score of the current target point to be inspected. The higher the score, the more urgent it is to inspect and photograph the target point to be inspected. Comprehensive evaluation value f: f = w1*D1+w2*D2+w3*P, where w iare preset weights, w1 is the preset geometric distance weight, w2 is the preset hazard level weight, and w3 is the preset device priority weight. These can be adjusted dynamically based on actual needs. D1 represents the geometric distance to the target point, D2 represents the regional hazard level of the target point to be inspected, and P represents the target device priority. A weighted calculation is performed based on the geometric distance between the current location and several target points to be inspected, the regional hazard level of each target point to be inspected, the target device priority, and the preset weights to obtain a comprehensive assessment value for each target point to be inspected.

[0088] S3. Collecting images of the target point to be inspected based on the comprehensive evaluation value, the initial inspection instruction, and the current shooting parameters; wherein the current shooting parameters during the initial acquisition are the initial shooting parameters;

[0089] Specifically, according to the comprehensive evaluation value, the initial inspection instruction and the current shooting parameters, collecting a photographic image of the target point to be inspected includes:

[0090] According to the initial inspection instruction, parsing the initial inspection route in the initial inspection instruction;

[0091] Rearranging the target points to be inspected on the initial inspection route according to the order of the comprehensive evaluation values ​​from large to small to generate a new inspection route;

[0092] Arrive at the corresponding shooting location according to the new inspection route, and adjust the posture, gimbal angle, and camera shooting parameters according to the current shooting parameters at the corresponding shooting location;

[0093] According to the adjusted posture, pan / tilt angle and camera shooting parameters, the camera collects images of the target points to be inspected.

[0094] In a preferred embodiment of the present invention, by reordering the target points on the initial inspection route to generate a new inspection route, and then analyzing the new inspection route, the drone can accurately understand the inspection path and target location, ensuring that the inspection mission is executed according to the predetermined plan. After arriving at the designated shooting location, the inspection drone adjusts its posture, gimbal angle, and camera shooting parameters based on the current shooting parameters, thereby accurately capturing image information of the target points to be inspected. Before shooting, the drone adjusts itself according to preset shooting parameters, including posture, gimbal angle, and camera settings, to ensure the clarity and accuracy of the captured image.

[0095] S4, inputting the captured image into a preset target recognition model to perform accuracy recognition, and obtaining the target accuracy of the captured image;

[0096] Preferably, the target recognition model is deployed on the inspection drone; the training of the target recognition model includes:

[0097] Obtain historical inspection images and target recognition results;

[0098] Marking the target edge, target position and target angle of the historical inspection image to obtain a target historical image;

[0099] Inputting the target historical image into the target recognition model to be trained, training according to the current model parameters, and obtaining a predicted recognition result; wherein the current model parameters at the initial time are the initial model parameters;

[0100] Calculating the recognition accuracy rate based on the predicted recognition results and the historical recognition results;

[0101] When the recognition accuracy is determined to have converged, the trained target recognition model is obtained;

[0102] When it is determined that the recognition accuracy has not converged, the current model parameters are updated according to the recognition accuracy, and the updated current model parameters are used as the current model parameters for the next training.

[0103] In a preferred embodiment of the present invention, the target recognition model can employ a deep learning target detection model, which is trained and deployed on the onboard computer of the inspection drone. By acquiring historical inspection images and target recognition results and using this data for training, the target recognition model can learn information such as target features, edges, position, and angle, thereby improving target recognition accuracy. During the training process, by continuously calculating the recognition accuracy based on predicted and historical recognition results and updating model parameters based on the recognition accuracy, the target recognition model gradually optimizes its recognition capabilities until the recognition accuracy converges. A drone equipped with the trained target recognition model can autonomously identify inspection targets without human intervention or assistance. Because the target recognition model can quickly and accurately identify inspection targets, the drone can quickly locate the target, reducing search and waiting time during the inspection process. This enables the drone to complete more inspection tasks in a shorter time, improving inspection efficiency. The target recognition model can be trained and adjusted according to different inspection scenarios and target types to meet diverse inspection requirements. Deployed on an inspection drone, it enables the drone to flexibly adapt to various complex inspection environments and targets, improving the adaptability and flexibility of inspections.

[0104] S5. Determine, based on the target accuracy and a preset threshold, whether the target accuracy is greater than the preset threshold;

[0105] In a preferred embodiment of the present invention, it is determined whether the target accuracy of the captured image is greater than a preset threshold, for example, it is determined whether the target accuracy is greater than 90%.

[0106] S6. If yes, transmit the captured image to the control center so that the control center stores the captured image;

[0107] In a preferred embodiment of the present invention, the inspection drone also captures the current position and inspection route corresponding to the image as waypoints and routes respectively, updates the waypoint data and route data, and transmits the updated waypoint data and route data to the control center for storage.

[0108] S7. If not, adjust the shooting distance and the deflection angle according to the target accuracy, and adjust the current shooting parameters according to the shooting distance and the deflection angle until the target accuracy is greater than the preset threshold, obtain a new shooting image, and transmit the new shooting image to the control center so that the control center stores the new shooting image.

[0109] Specifically, adjusting the shooting distance and the deflection angle according to the target accuracy, and adjusting the current shooting parameters according to the shooting distance and the deflection angle include:

[0110] Get the current location;

[0111] Determining, according to the target accuracy, a first size, a first position, and a first angle of the corresponding target point to be inspected in the captured image;

[0112] Calculating a shooting distance between a target point to be inspected and a current position according to the first size and the first position; and adjusting the shooting distance according to the first size and the first position;

[0113] Calculating a deflection angle between the target point to be inspected and the current position based on the first angle and the first position; and adjusting the deflection angle based on the first angle and the first position;

[0114] According to the adjusted shooting distance and the adjusted deflection angle, adjust the current shooting parameters' own posture, gimbal angle, and camera shooting parameters.

[0115] In a preferred embodiment of the present invention, based on the position (first position) and angle (first angle) of the target in the image, combined with the current orientation and position information of the drone, the angle at which the drone needs to be deflected is calculated so as to more accurately aim at the target. Based on the calculated shooting distance and deflection angle, the drone's own posture (such as altitude, forward / backward, left and right movement, etc.), gimbal angle (pitch, yaw, roll, etc.) and camera shooting parameters (such as focal length, aperture, shutter speed, etc.) are adjusted to ensure that the target is presented clearly and accurately in the image. For example, the size, position and angle of the device in the captured image are determined. If the size is small, the drone moves forward, otherwise it moves backward; if the device position is biased to the right, the drone moves right or turns right; if the angle is deflected, the aircraft and gimbal angles are adjusted accordingly.

[0116] In another preferred embodiment of the present invention, the inspection drone communicates with the control center through the deployed communication module, such as Figure 2 As shown. The route planning module of the control center is used to generate an inspection route and generate an initial inspection instruction, which is transmitted to the inspection drone through the communication module. After receiving the initial inspection instruction, the inspection drone collects the captured image of the target point to be inspected according to the initial inspection instruction and the current shooting parameters through the image acquisition module, and inputs the captured image into the preset target recognition model through the image recognition module for accuracy recognition to obtain the target accuracy of the captured image. In the real-time photo analysis module, according to the target accuracy and the preset threshold, it is determined whether the target accuracy is greater than the preset threshold. If so, the captured image is transmitted to the control center through the communication module so that the control center stores the captured image; if not, the shooting posture correction module adjusts the current shooting parameters according to the target accuracy until the target accuracy is greater than the preset threshold, and a new captured image is obtained, and the new captured image is transmitted to the control center through the communication module so that the control center stores the new captured image. The algorithm based on real-time image recognition can only guarantee the shooting quality of the target device in the picture during shooting, but cannot guarantee the quality of the photos taken. The real-time photo analysis module can directly analyze the photos taken, fully considering the impact of various variables before, during and after shooting on the actual shooting effect; the photo-based drone shooting posture correction module comprehensively considers external environmental disturbances and position errors to achieve precise alignment of the target point and high-quality shooting.

[0117] By implementing this embodiment, the geometric distance between the current position and several target points to be inspected, the regional hazard level of each target point to be inspected, the target device priority, the initial inspection instruction transmitted by the control center, and the initial shooting parameters are obtained; based on the geometric distance, the regional hazard level, and the target device priority, a weighted calculation is performed to obtain a comprehensive evaluation value of each target point to be inspected; based on the comprehensive evaluation value, the initial inspection instruction, and the current shooting parameters, a captured image of the target point to be inspected is collected; wherein the current shooting parameters at the time of initial collection are the initial shooting parameters; the captured image is input into a preset target recognition model for accuracy recognition to obtain a target accuracy of the captured image; based on the target accuracy and a preset threshold, it is determined whether the target accuracy is greater than a preset threshold; if so, the captured image is transmitted to the control center so that the control center stores the captured image; if not, the shooting distance and deflection angle are adjusted according to the target accuracy, and the current shooting parameters are adjusted according to the shooting distance and deflection angle until the target accuracy is greater than the preset threshold, thereby obtaining a new captured image, and the new captured image is transmitted to the control center so that the control center stores the new captured image. The drone can automatically collect and shoot images based on the initial inspection instructions of the control center, the current shooting parameters and the comprehensive evaluation values ​​of each target point to be inspected, reducing human intervention; the accuracy of the captured images is recognized through the preset target recognition model, and the current shooting parameters are adjusted according to the target accuracy so that the target accuracy is greater than the preset threshold, ensuring that the collected images have a high recognition accuracy. Even if the target accuracy does not reach the preset threshold due to the influence of the external environment, the drone can dynamically adjust its own shooting distance and deflection angle, and then adjust the current shooting parameters according to the adjusted shooting distance and deflection angle to obtain high-quality photos, thereby improving the accuracy of images taken during inspections.

[0118] See also Figure 3 , is a structural diagram of a UAV autonomous inspection system provided by one embodiment of the present invention, including an inspection UAV and a control center;

[0119] The inspection drone is used to obtain the geometric distance between the current position and several target points to be inspected, the regional hazard level of each target point to be inspected, the target device priority, the initial inspection instruction transmitted by the control center, and the initial shooting parameters; based on the geometric distance, the regional hazard level, and the target device priority, a weighted calculation is performed to obtain a comprehensive evaluation value of each target point to be inspected; based on the comprehensive evaluation value, the initial inspection instruction, and the current shooting parameters, a photographic image of the target point to be inspected is collected; wherein the current shooting parameters at the time of initial collection are the initial shooting parameters; the photographic image is input into a preset target recognition model for accuracy recognition to obtain a target accuracy of the photographic image; based on the target accuracy and a preset threshold, it is determined whether the target accuracy is greater than a preset threshold; if so, the photographic image is transmitted to the control center so that the control center stores the photographic image; if not, the shooting distance and deflection angle are adjusted according to the target accuracy, and the current shooting parameters are adjusted according to the shooting distance and deflection angle until the target accuracy is greater than the preset threshold, thereby obtaining a new photographic image, and the new photographic image is transmitted to the control center so that the control center stores the new photographic image;

[0120] The control center is used to store the captured images transmitted by the inspection drone and new captured images.

[0121] Specifically, the control center is further configured to generate an initial inspection instruction and transmit the initial inspection instruction to the inspection drone;

[0122] The generation of the initial inspection instruction includes:

[0123] Obtain route data, waypoint data, target point location data, and target equipment information of the target point to be inspected;

[0124] Performing path planning based on the route data, the waypoint data, the target point location data, and the target device information to generate an initial inspection route;

[0125] An initial inspection instruction is generated according to the initial inspection route.

[0126] Preferably, the inspection drone is further configured to collect images of the target point to be inspected based on the comprehensive evaluation value, the initial inspection instruction, and current shooting parameters;

[0127] According to the initial inspection instruction and the current shooting parameters, an image of the target point to be inspected is collected, including:

[0128] According to the initial inspection instruction, parsing the initial inspection route in the initial inspection instruction;

[0129] Rearranging the target points to be inspected on the initial inspection route according to the order of the comprehensive evaluation values ​​from large to small to generate a new inspection route;

[0130] Arrive at the corresponding shooting location according to the new inspection route, and adjust the posture, gimbal angle, and camera shooting parameters according to the current shooting parameters at the corresponding shooting location;

[0131] According to the adjusted posture, pan / tilt angle and camera shooting parameters, the camera collects images of the target points to be inspected.

[0132] The present invention provides an autonomous inspection system for unmanned aerial vehicles (UAVs), which can automatically collect and shoot images according to the initial inspection instructions of the control center, current shooting parameters and comprehensive evaluation values ​​of each target point to be inspected, thereby reducing manual intervention; the accuracy of the shot images is recognized by a preset target recognition model, and the current shooting parameters are adjusted according to the target accuracy so that the target accuracy is greater than a preset threshold, thereby ensuring that the collected shot images have a high recognition accuracy; even if the target accuracy does not reach the preset threshold due to the influence of the external environment, the shooting distance and deflection angle of the UAV can be dynamically adjusted, and then the current shooting parameters are adjusted according to the adjusted shooting distance and deflection angle to obtain high-quality photos, thereby improving the accuracy of the images shot during inspections.

[0133] It should be noted that the system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the system embodiment provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0134] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system described above may refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0135] Another embodiment of the present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the drone autonomous inspection method described in the above embodiment.

[0136] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. The computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium.

[0137] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A drone autonomous inspection method, characterized in that: An inspection drone suitable for an autonomous drone inspection system; the autonomous drone inspection system also includes a control center; the autonomous drone inspection method includes: Obtaining the geometric distance between the current position and several target points to be inspected, the regional hazard level of each target point to be inspected, the priority of the target device, the initial inspection instruction transmitted by the control center, and the initial shooting parameters; A comprehensive evaluation value of each target point to be inspected is obtained by weighted calculation based on the geometric distance, the regional danger level, and the target device priority; According to the comprehensive evaluation value, the initial inspection instruction and the current shooting parameters, the image of the target point to be inspected is collected; wherein the current shooting parameters at the time of initial collection are the initial shooting parameters; Inputting the captured image into a preset target recognition model for accuracy recognition to obtain the target accuracy of the captured image; According to the target accuracy rate and a preset threshold, determining whether the target accuracy rate is greater than the preset threshold rate; If yes, transmitting the captured image to the control center so that the control center stores the captured image; If not, adjust the shooting distance and deflection angle according to the target accuracy, and adjust the current shooting parameters according to the shooting distance and deflection angle until the target accuracy is greater than the preset threshold, obtain a new shooting image, and transmit the new shooting image to the control center so that the control center stores the new shooting image.

2. The autonomous inspection method of a drone according to claim 1, characterized in that: The control center is used to generate an initial inspection instruction and transmit the initial inspection instruction to the inspection drone; The generation of the initial inspection instruction includes: Obtain route data, waypoint data, target point location data, and target equipment information of the target point to be inspected; Performing path planning based on the route data, the waypoint data, the target point location data, and the target device information to generate an initial inspection route; An initial inspection instruction is generated according to the initial inspection route.

3. The autonomous inspection method of a drone according to claim 2, characterized in that: According to the comprehensive evaluation value, the initial inspection instruction and the current shooting parameters, a photographic image of the target point to be inspected is collected, including: According to the initial inspection instruction, parsing the initial inspection route in the initial inspection instruction; Rearranging the target points to be inspected on the initial inspection route according to the order of the comprehensive evaluation values ​​from large to small to generate a new inspection route; Arrive at the corresponding shooting location according to the new inspection route, and adjust the posture, gimbal angle, and camera shooting parameters according to the current shooting parameters at the corresponding shooting location; According to the adjusted posture, pan / tilt angle and camera shooting parameters, the camera collects images of the target points to be inspected.

4. The autonomous inspection method of a drone according to claim 3, characterized in that: The target recognition model is deployed on the inspection drone; The training of the target recognition model includes: Obtain historical inspection images and target recognition results; Marking the target edge, target position and target angle of the historical inspection image to obtain a target historical image; Inputting the target historical image into the target recognition model to be trained, training according to the current model parameters, and obtaining a predicted recognition result; wherein the current model parameters at the initial time are the initial model parameters; Calculating the recognition accuracy rate based on the predicted recognition results and the historical recognition results; When the recognition accuracy is determined to have converged, the trained target recognition model is obtained; When it is determined that the recognition accuracy has not converged, the current model parameters are updated according to the recognition accuracy, and the updated current model parameters are used as the current model parameters for the next training.

5. The autonomous inspection method of a drone according to claim 4, characterized in that: Adjusting the shooting distance and the deflection angle according to the target accuracy, and adjusting the current shooting parameters according to the shooting distance and the deflection angle, including: Get the current location; Determining, according to the target accuracy, a first size, a first position, and a first angle of the corresponding target point to be inspected in the captured image; Calculating a shooting distance between a target point to be inspected and a current position according to the first size and the first position; and adjusting the shooting distance according to the first size and the first position; Calculating a deflection angle between the target point to be inspected and the current position based on the first angle and the first position; and adjusting the deflection angle based on the first angle and the first position; According to the adjusted shooting distance and the adjusted deflection angle, adjust the current shooting parameters' own posture, gimbal angle, and camera shooting parameters.

6. The autonomous inspection method of a drone according to claim 5, characterized in that: The control center is configured to perform path planning based on the route data, the waypoint data, the target point location data, and the target device information to generate an initial inspection route; The route data includes: route number and route type; The waypoint data includes: the longitude and latitude of the waypoint and the altitude of the waypoint; The target point location data includes: the target point latitude and longitude and the target point height of the target point to be inspected; The target device information includes: the target device priority and device height of the target point to be inspected; Performing path planning based on the route data, the waypoint data, the target point location data, and the target device information to generate an initial inspection route includes: Matching the corresponding route type and waypoint data according to the route number; and determining the spatial correspondence between the longitude and latitude of the waypoint and the longitude and latitude of the target point according to the matched waypoint data; According to the spatial correspondence, based on the waypoint altitude, the target point altitude and the device altitude, a reference flight altitude is calculated to meet the flight requirements; Generate an initial target route using the matched waypoint data as path nodes and the reference flight altitude; and map the target point location data to the initial target route to generate an inspection path segment; The inspection path segments are weighted according to the priority of the target device, so that the positions of the inspection target points are sorted in descending order of the priority of the target device, and the initial inspection route is obtained.

7. An autonomous inspection system for drones, characterized in that: Including inspection drones and control centers; The inspection drone is used to obtain the geometric distance between the current position and several target points to be inspected, the regional hazard level of each target point to be inspected, the target device priority, the initial inspection instruction transmitted by the control center, and the initial shooting parameters; based on the geometric distance, the regional hazard level, and the target device priority, a weighted calculation is performed to obtain a comprehensive evaluation value of each target point to be inspected; based on the comprehensive evaluation value, the initial inspection instruction, and the current shooting parameters, a photographic image of the target point to be inspected is collected; wherein the current shooting parameters at the time of initial collection are the initial shooting parameters; the photographic image is input into a preset target recognition model for accuracy recognition to obtain a target accuracy of the photographic image; based on the target accuracy and a preset threshold, it is determined whether the target accuracy is greater than a preset threshold; if so, the photographic image is transmitted to the control center so that the control center stores the photographic image; if not, the shooting distance and deflection angle are adjusted according to the target accuracy, and the current shooting parameters are adjusted according to the shooting distance and deflection angle until the target accuracy is greater than the preset threshold, thereby obtaining a new photographic image, and the new photographic image is transmitted to the control center so that the control center stores the new photographic image; The control center is used to store the captured images transmitted by the inspection drone and new captured images.

8. The autonomous inspection system for unmanned aerial vehicles according to claim 7, characterized in that: The control center is further configured to generate an initial inspection instruction and transmit the initial inspection instruction to the inspection drone; The generation of the initial inspection instruction includes: Obtain route data, waypoint data, target point location data, and target equipment information of the target point to be inspected; Performing path planning based on the route data, the waypoint data, the target point location data, and the target device information to generate an initial inspection route; An initial inspection instruction is generated according to the initial inspection route.

9. The autonomous inspection system for unmanned aerial vehicles according to claim 8, characterized in that: The inspection drone is further configured to collect images of the target point to be inspected based on the comprehensive evaluation value, the initial inspection instruction, and the current shooting parameters; According to the initial inspection instruction and the current shooting parameters, an image of the target point to be inspected is collected, including: According to the initial inspection instruction, parsing the initial inspection route in the initial inspection instruction; Rearranging the target points to be inspected on the initial inspection route according to the order of the comprehensive evaluation values ​​from large to small to generate a new inspection route; Arrive at the corresponding shooting location according to the new inspection route, and adjust the posture, gimbal angle, and camera shooting parameters according to the current shooting parameters at the corresponding shooting location; According to the adjusted posture, pan / tilt angle and camera shooting parameters, the camera collects images of the target points to be inspected.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the drone autonomous inspection method according to any one of claims 1 to 6.

Images