Low-altitude unmanned aerial vehicle intelligent inspection system and method based on deep learning

Abstract

The invention discloses a low-altitude unmanned aerial vehicle intelligent inspection system and method based on deep learning, and relates to the technical field of unmanned aerial vehicle intelligent control. An unmanned aerial vehicle information set and a task information set are collected in advance, an anomaly detection model is constructed based on the task information set, and unmanned aerial vehicle real-time flight parameter data are collected; based on the anomaly detection model, the real-time flight parameter data and the unmanned aerial vehicle information set, constructing a parameter adjustment optimization model, solving and outputting a flight parameter adjustment strategy, sending the flight parameter adjustment strategy to the unmanned aerial vehicle, and collecting anomaly detection information collected in the flight process of the unmanned aerial vehicle in real time; performing dynamic parameter adjustment; and the intelligent level and the operation efficiency of unmanned aerial vehicle inspection are obviously improved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) intelligent control technology, and in particular to a low-altitude UAV intelligent inspection system and method based on deep learning. Background Art

[0002] Low-altitude drone inspections have become an efficient tool in modern infrastructure monitoring and agricultural management. During the inspection process, drones must flexibly respond to dynamic changes in the environment and various abnormal situations. Abnormalities are usually uncertain and may change in location, form, and severity, such as damaged power lines, tiny cracks in bridges, and pests and diseases in farmland. These abnormalities are often not fixed, and traditional static flight parameter settings cannot cope with such complex and changing inspection task requirements.

[0003] Existing inspection technologies often rely on pre-set flight paths and parameters, and these static configurations are significantly inadequate in the face of real-time changes. Static parameters can cause drones to miss critical details when detecting anomalies because they are unable to adjust their flight altitude, speed, and path based on emerging anomalies. Static path planning can also lead to wasted energy and reduced inspection efficiency because it cannot avoid obstacles or prioritize important areas in real time. In addition, static parameter settings ignore real-time changes in environmental factors such as weather, wind speed, and light, which can significantly affect the inspection effect and safety of drones.

[0004] A Chinese patent with authorization announcement number CN116071962B discloses an automatic highway drone inspection system, including a drone airport for realizing drone takeoff and landing operations, an information collection module for collecting image information and position information of a road section to be inspected, a highway maintenance operation platform for estimating the repair time of a diseased road section based on the image information of the road section to be inspected, and for matching the position information of the diseased road section with the corresponding estimated repair time, a task management module for replanning the drone inspection route based on the position information of the diseased road section and the corresponding estimated repair time information, and replanning the inspection route through the joint action of data measurement, parameter calculation and estimation of the repair time of the diseased road section; however, this method fails to solve the problem of dynamic parameter control during flight.

[0005] To this end, the present invention proposes a low-altitude UAV intelligent inspection system and method based on deep learning. Summary of the invention

[0006] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a low-altitude UAV intelligent inspection system and method based on deep learning, which significantly improves the intelligence level and operation efficiency of UAV inspection.

[0007] To achieve the above objectives, a low-altitude UAV intelligent inspection method based on deep learning is proposed, which includes the following steps:

[0008] Step 1: Collect drone information set and mission information set in advance;

[0009] Step 2: Build an anomaly detection model based on the task information set;

[0010] Step 3: Collect the real-time flight parameter data of the UAV, build a parameter adjustment optimization model based on the anomaly detection model, real-time flight parameter data and UAV information set, and solve the output flight parameter adjustment strategy;

[0011] Step 4: Send flight parameter adjustment strategies to the drone and collect anomaly detection information collected during the drone flight in real time;

[0012] Step 5: Dynamically adjust parameters based on anomaly detection information and drone information collection.

[0013] The drone information set is collected in the following manner:

[0014] According to the actual hardware configuration of the drone, various hardware parameters of the drone are counted as the drone information set;

[0015] The task information set is collected in the following manner:

[0016] Collect all preset inspection scenarios and corresponding inspection task types;

[0017] For each type of inspection task in each inspection scenario, sample images with labeled labels and position annotations are collected in advance to form a task information set.

[0018] The method of constructing anomaly detection model based on task information set is:

[0019] For each inspection scenario, the corresponding inspection task type is:

[0020] Read a sample image set corresponding to the inspection task type from the task information set;

[0021] Divide the sample image set into training set, validation set and test set;

[0022] Select the YOLO model or YOLOv3 model as the anomaly detection model;

[0023] Resize all sample images to a uniform size to fit the input requirements of the anomaly detection model;

[0024] Apply data augmentation techniques such as random rotation, flipping, and scaling to sample images to improve the generalization ability of the model;

[0025] Preset the hyperparameter set of the anomaly detection model;

[0026] The anomaly detection model uses the weighted sum of the positioning loss of bounding box regression, the classification loss of cross entropy loss, and the confidence loss of target box matching as the loss function;

[0027] Use the training set to train the anomaly detection model and perform forward propagation to calculate the prediction results;

[0028] Calculate the loss and optimize the model parameters of the anomaly detection model through back-propagation;

[0029] Evaluate the model on the validation set, check for overfitting, and manually adjust hyperparameters until the anomaly detection model reaches the preset accuracy on the validation set;

[0030] The output of the anomaly detection model includes bounding box information, confidence scores, and category labels for each detection box in the sample image;

[0031] The anomaly detection model is further used on the test set to verify the model performance, and the accuracy, recall and F1 score are recorded.

[0032] The method of collecting real-time flight parameter data of the drone is:

[0033] According to the needs, set the actual inspection scene and inspection task type for the drone in the drone background;

[0034] When controlling the drone to actually perform inspection tasks, the drone's built-in image capture equipment, sensors and navigation system are used to collect various flight parameters in real time to form real-time flight parameter data.

[0035] The method of constructing a parameter adjustment optimization model based on anomaly detection model, real-time flight parameter data and drone information set, and solving the output flight parameter adjustment strategy includes the following steps:

[0036] Step 31: Input the real-time image collected by the image capture device of the drone into the anomaly detection model in real time. If the confidence of the output of the anomaly detection model is greater than the preset confidence threshold, it is determined that an anomaly exists and the process goes to step 32; if the output confidence is less than or equal to the preset confidence threshold, no processing is performed;

[0037] Step 32: Determine the actual geographic coordinates of the abnormal location where the abnormality occurs in the inspection scene according to the position of the bounding box output by the anomaly detection model in the real-time image and the real-time position of the drone;

[0038] The method for determining the actual geographic coordinates of the abnormal location where the abnormality occurs in the inspection scene is:

[0039] Get the anomaly bounding box data (x) from the output of the YOLO model bbox ,y bbox , w bbox ,h bbox ), where x bbox Indicates the central horizontal coordinate of the bounding box, y bbox represents the center ordinate of the bounding box, w bbox Indicates the width of the bounding box, h bbox Indicates the height of the bounding box;

[0040] Mark the spatial coordinates of the drone's current position as (xc, yc, zc);

[0041] Use the drone's flight altitude zc and the camera's viewing angle parameters to convert the image plane coordinates into ground coordinates;

[0042] Denote the viewing angle of the drone’s image capture device as θ, and calculate the conversion coefficient SC from pixels to actual distance;

[0043] According to the central horizontal coordinate and the conversion coefficient, the geographic horizontal coordinate xr of the abnormal position is calculated;

[0044] According to the central ordinate and the conversion coefficient, calculate the geographic ordinate yr of the abnormal position;

[0045] Then the actual geographic coordinates of the abnormal position are expressed as (geographic abscissa, geographic ordinate);

[0046] Step 33: Calculate the anomaly impact factor e according to the confidence output by the anomaly detection model and the actual coverage area of ​​the anomaly position;

[0047] Step 34: Calculate the abnormal distance factor d according to the real-time position of the UAV and the abnormal position;

[0048] Step 35: Calculate the path adjustment proportional coefficient g according to the current real-time flight speed and flight altitude of the UAV;

[0049] Step 36: Adjust the flight speed according to the abnormal impact factor e;

[0050] Step 37: adjusting the flight altitude according to the abnormal distance factor d;

[0051] Step 38: adjusting the path offset ΔP according to the path adjustment proportional coefficient g; the calculation formula of the path offset ΔP is: ΔP=g×(xc-xr,yc-yr);

[0052] Step 39: The abnormal impact factor e, the abnormal distance factor d and the path adjustment proportional coefficient g are used to form a flight parameter adjustment strategy.

[0053] The method of real-time collection of abnormal detection information collected during the flight of the drone is:

[0054] During the inspection process after the drone is adjusted according to the flight parameter adjustment strategy, the abnormal position, abnormal category, confidence level, and abnormal area continue to be collected as abnormality detection information;

[0055] The method of dynamic parameter adjustment based on anomaly detection information and drone information collection is as follows:

[0056] If the difference between the real-time confidence and the confidence at the previous unit time is greater than the preset confidence difference threshold, or the difference between the area covered by the real-time abnormal position and the area covered by the abnormal position at the previous unit time is greater than the preset area difference threshold, the flight speed and flight altitude are dynamically adjusted.

[0057] A low-altitude UAV intelligent inspection system based on deep learning is proposed, which includes an information collection module, a detection model construction module, a strategy static adjustment module and a strategy dynamic adjustment module; wherein each module is electrically connected;

[0058] An information collection module collects drone information sets and mission information sets in advance, sends the mission information sets to the detection model building module, and sends the drone information sets to the strategy static adjustment module;

[0059] The detection model building module builds an anomaly detection model based on the task information set and sends the anomaly detection model to the strategy static adjustment module;

[0060] The strategy static adjustment module collects the real-time flight parameter data of the UAV, builds a parameter adjustment optimization model based on the anomaly detection model, the real-time flight parameter data and the UAV information set, solves and outputs the flight parameter adjustment strategy, sends the flight parameter adjustment strategy to the UAV, and collects the anomaly detection information collected during the flight of the UAV in real time, and sends the collected anomaly detection information to the strategy dynamic adjustment module;

[0061] The strategy dynamic adjustment module performs dynamic parameter adjustment based on anomaly detection information and drone information collection.

[0062] An electronic device is proposed, comprising: a processor and a memory, wherein the memory stores a computer program that can be called by the processor;

[0063] The processor executes the above-mentioned low-altitude UAV intelligent inspection method based on deep learning by calling the computer program stored in the memory.

[0064] A computer-readable storage medium is provided, on which a rewritable computer program is stored;

[0065] When the computer program runs on a computer device, the computer device executes the above-mentioned low-altitude UAV intelligent inspection method based on deep learning.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] The present invention collects the information set of unmanned aerial vehicles and the task information set in advance, builds an anomaly detection model based on the task information set, collects the real-time flight parameter data of the unmanned aerial vehicles, builds a parameter adjustment optimization model based on the anomaly detection model, the real-time flight parameter data and the unmanned aerial vehicle information set, solves and outputs the flight parameter adjustment strategy, sends the flight parameter adjustment strategy to the unmanned aerial vehicle, and collects the anomaly detection information collected during the flight of the unmanned aerial vehicle in real time, and performs dynamic parameter adjustment based on the anomaly detection information and the unmanned aerial vehicle information set; uses the deep learning model to detect and identify anomalies in real time, and dynamically adjusts the flight parameters of the unmanned aerial vehicle in combination with the hardware and software information of the unmanned aerial vehicle, such as camera resolution, navigation accuracy and endurance. This method can not only ensure that the unmanned aerial vehicle maintains a suitable observation angle and flight path during the inspection process, but also improve the detection efficiency and accuracy. By integrating real-time data processing and parameter adjustment algorithms, the unmanned aerial vehicle can automatically optimize its inspection tasks according to environmental changes, meet diverse inspection needs, and significantly improve the intelligence level and operation efficiency of unmanned aerial vehicle inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 This is a flow chart of a low-altitude UAV intelligent inspection method based on deep learning in Example 1 of the present invention;

[0069] Figure 2 This is a module connection diagram of the low-altitude UAV intelligent inspection system based on deep learning in Example 2 of the present invention. DETAILED DESCRIPTION

[0070] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] Example 1

[0072] like Figure 1 As shown in the figure, the low-altitude UAV intelligent inspection method based on deep learning includes the following steps:

[0073] Step 1: Collect drone information set and mission information set in advance;

[0074] Step 2: Build an anomaly detection model based on the task information set;

[0075] Step 3: Collect the real-time flight parameter data of the UAV, build a parameter adjustment optimization model based on the anomaly detection model, real-time flight parameter data and UAV information set, and solve the output flight parameter adjustment strategy;

[0076] Step 4: Send flight parameter adjustment strategies to the drone and collect anomaly detection information collected during the drone flight in real time;

[0077] Step 5: Dynamically adjust parameters based on anomaly detection information and drone information collection.

[0078] The drone information set is collected in the following manner:

[0079] According to the actual hardware configuration of the drone, various hardware parameters of the drone are counted as the drone information set;

[0080] Specifically, the drone information set includes hardware information such as camera resolution, sensor type and accuracy, maximum flight time, maximum flight speed and maximum load capacity of the drone, as well as software information such as navigation system accuracy and real-time data processing capabilities.

[0081] Furthermore, the task information set is collected in the following manner:

[0082] Collect all preset inspection scenarios and corresponding inspection task types; it is understandable that the specific inspection tasks that need to be performed are different for different inspection scenarios, for example: the inspection scenarios may include power lines, bridges, farmlands, etc., and the inspection task types corresponding to each inspection scenario may be damage, cracks, pests and diseases, etc., that is, when the inspection scenario is a power line, drone inspection can be used to detect whether the line is damaged. Similarly, when the bridge is used as the inspection scenario, drone inspection can be used to detect whether the bridge has cracks. When the farmland is used as the inspection scenario, drone inspection can be used to detect whether the crops have pests and diseases. Therefore, in different inspection tasks, the abnormal situations that the drone needs to identify are different; it should be noted that the above-mentioned inspection scenarios listed here in this embodiment are only some typical examples. In actual application, they can be further expanded. The expansion of inspection scenarios and the expansion of inspection tasks are within the protection scope of the present invention.

[0083] For each type of inspection task in each inspection scenario, several sample images with labeled labels and position annotations are collected in advance to form a task information set;

[0084] It should be noted that the collected sample images should ensure that the sample data covers different environments (such as weather conditions, seasonal changes) and perspectives, and it is best to use image data collected from historical inspection tasks, public data sets or manual shooting.

[0085] Specifically, the label annotation refers to annotating the existing anomalies in the sample image, and the anomaly is the anomaly type corresponding to the inspection task, such as damage, cracks, pests and diseases, etc.; for sample images with abnormalities, the label annotation is set to 1, and for sample images without abnormalities, the label annotation is set to 0.

[0086] The position annotation refers to providing abnormal bounding box information for a sample image with an abnormality, and the bounding box information includes the coordinates of the midline point of the bounding box, and the width and height of the bounding box.

[0087] Furthermore, the method of constructing an anomaly detection model based on the task information set is:

[0088] For each inspection scenario, the corresponding inspection task type is:

[0089] Read a sample image set corresponding to the inspection task type from the task information set;

[0090] Divide the sample image set into a training set, a validation set, and a test set; generally, the ratios of the training set, the validation set, and the test set are 70%, 15%, and 15%;

[0091] Select the YOLO model or YOLOv3 model as the anomaly detection model;

[0092] Resize all sample images to a uniform size to fit the input requirements of the anomaly detection model;

[0093] Apply data augmentation techniques such as random rotation, flipping, and scaling to sample images to improve the generalization ability of the model;

[0094] Presetting a set of hyperparameters of anomaly detection models; the set of hyperparameters includes learning rate, batch size, number of training rounds, etc.;

[0095] The anomaly detection model uses the weighted sum of the positioning loss of bounding box regression, the classification loss of cross entropy loss, and the confidence loss of target box matching as the loss function;

[0096] Use the training set to train the anomaly detection model and perform forward propagation to calculate the prediction results;

[0097] Calculate the loss and optimize the model parameters of the anomaly detection model through back-propagation;

[0098] Evaluate the model on the validation set, check for overfitting, and manually adjust hyperparameters until the anomaly detection model reaches the preset accuracy on the validation set;

[0099] The output of the anomaly detection model includes bounding box information, confidence scores, and category labels for each detection box in the sample image;

[0100] The anomaly detection model is further used on the test set to verify the model performance, and the accuracy, recall and F1 score are recorded.

[0101] Furthermore, the method of collecting real-time flight parameter data of the drone is:

[0102] According to the needs, set the actual inspection scene and inspection task type for the drone in the drone background;

[0103] When controlling the drone to actually perform inspection tasks, the drone's built-in image capture devices, sensors and navigation systems, such as the onboard camera, IMU (inertial measurement unit) and GPS module, are used to collect various flight parameters in real time to form real-time flight parameter data.

[0104] The flight parameters include: real-time images captured by the image capture device, the current flight speed captured by the sensor and navigation system, the current flight altitude, GPS coordinates including longitude, latitude and altitude, attitude information including pitch angle, yaw angle and roll angle, battery status including power percentage, discharge speed, etc.

[0105] Furthermore, the construction of a parameter adjustment optimization model based on the anomaly detection model, the real-time flight parameter data and the drone information set, and the solution of the output flight parameter adjustment strategy include the following steps:

[0106] Step 31: Input the real-time image collected by the image capture device of the drone into the anomaly detection model in real time. If the confidence of the output of the anomaly detection model is greater than the preset confidence threshold, it is determined that an anomaly exists and the process goes to step 32; if the output confidence is less than or equal to the preset confidence threshold, no processing is performed;

[0107] Step 32: Determine the actual geographic coordinates of the abnormal location where the abnormality occurs in the inspection scene according to the position of the bounding box output by the anomaly detection model in the real-time image and the real-time position of the drone;

[0108] Preferably, the method of determining the actual geographic coordinates of the abnormal location where the abnormality occurs in the inspection scene is:

[0109] Get the anomaly bounding box data (x) from the output of the YOLO model bbox ,y bbox , w bbox ,hbbox ), where x bbox Indicates the central horizontal coordinate of the bounding box, y bbox represents the center ordinate of the bounding box, w bbox Indicates the width of the bounding box, h bbox Indicates the height of the bounding box;

[0110] Mark the spatial coordinates of the drone's current position as (xc, yc, zc);

[0111] Use the drone's flight altitude zc and the camera's viewing angle parameters such as focal length and field of view to convert the image plane coordinates into ground coordinates;

[0112] The viewing angle of the drone's image capture device is marked as θ, and the conversion coefficient SC from pixels to actual distance is calculated; the conversion coefficient SC is calculated as follows: Among them, Iw represents the width of the image captured by the image capture device, in pixels; it is used to convert the image coordinates of the bounding box into actual geographic coordinates in the field of view of the drone; since the drone camera can be simplified to a pinhole camera model, this is a commonly used model in computer vision. This model assumes that all light passes through a single point and forms an image on the sensor, and this image is a two-dimensional projection of the three-dimensional world. The size of the object in the image depends on the distance from the camera. The closer the object is to the camera, the larger it appears in the image. This change is described by the perspective projection relationship, and the height directly affects the actual distance represented by a pixel on the ground. For a fixed field of view angle, the higher the flight altitude, the larger the actual area covered by a pixel on the ground; and the viewing angle θ is the maximum viewing angle range that the camera lens can see. The field of view angle determines the width of the ground area that the camera can see; therefore, based on the flight altitude and viewing angle, based on the similarity of triangles, the conversion coefficient between coordinate systems can be derived;

[0113] According to the central horizontal coordinate and the conversion coefficient, the geographic horizontal coordinate xr of the abnormal position is calculated; specifically, the calculation formula of the geographic horizontal coordinate xr is:

[0114] According to the central ordinate and the conversion coefficient, the geographic ordinate yr of the abnormal position is calculated; specifically, the calculation formula of the geographic ordinate yr is: Ih represents the height of the image captured by the image capture device, in pixels;

[0115] The actual geographic coordinates of the abnormal position are expressed as (geographic horizontal coordinate, geographic vertical coordinate). Since the target location of a general inspection task is the ground or a lower scene, the actual height of the abnormal position can be set to 0.

[0116] Step 33: Calculate the anomaly impact factor e based on the confidence output by the anomaly detection model and the actual coverage area of ​​the anomaly location; the actual coverage area is the mapping of the bounding box in the actual geographic coordinates, that is, the area can be calculated as w bbox ×SC×h bbox ×SC; the calculation formula of the abnormal impact factor e can be: e=c×s×t, where c is the confidence level, s is the actual coverage area, and t is the impact weight of the preset inspection task type. The more important the inspection task, the larger the t value;

[0117] Step 34: Calculate the abnormal distance factor d according to the real-time position and abnormal position of the UAV; specifically, the calculation formula of the abnormal distance factor d is: d=zc;

[0118] Step 35: Calculate the path adjustment proportional coefficient g according to the current real-time flight speed and flight altitude of the UAV; the calculation formula of the path adjustment proportional coefficient g is: Among them, k is the preset sensitivity constant, v is the real-time flight speed of the UAV, and h is the real-time altitude of the UAV;

[0119] Step 36: According to the abnormality impact factor e, the flight speed is adjusted; the flight speed is adjusted in the following manner: the flight speed v is adjusted to v-k1×e; k1 is a preset proportional coefficient; it can be understood that when the confidence level and the range covered by the abnormality are larger or the impact weight is larger, the abnormality is more serious, and the flight speed needs to be reduced to ensure that the UAV can fully observe and confirm the abnormality;

[0120] Step 37: adjusting the flight height according to the abnormal distance factor d; the flight height is adjusted in the following manner: adjusting the flight height zc to zc-k2×d, where k2 is a preset proportional coefficient; it can be understood that the farther the distance to the abnormality is, the greater the amplitude of the height reduction is, so as to be closer to the abnormal position, thereby ensuring higher clarity and resolution of observation;

[0121] Step 38: According to the path adjustment proportional coefficient g, the path offset ΔP is adjusted; the calculation formula of the path offset ΔP is: ΔP = g × (xc-xr, yc-yr); it can be understood that the path offset ΔP refers to offsetting the flight path of the drone to the location of the abnormal position. The greater the flight speed or the lower the flight altitude, the smaller the offset is, so as to prevent excessive adjustment from passing over the abnormal position and causing omission of observation of the abnormality;

[0122] Step 39: The abnormal impact factor e, the abnormal distance factor d and the path adjustment proportional coefficient g are used to form a flight parameter adjustment strategy.

[0123] Furthermore, sending the flight parameter adjustment strategy to the drone is to send the flight parameter adjustment strategy to the drone control system, and the drone control system regulates the flight speed, flight altitude and flight path of the drone according to the flight parameter adjustment strategy.

[0124] The method of collecting the abnormal detection information collected during the flight of the drone in real time is:

[0125] During the inspection process after the drone is adjusted according to the flight parameter adjustment strategy, the abnormal position, abnormal category, confidence level, and abnormal area continue to be collected as abnormality detection information;

[0126] It is understandable that after the drone adjusts its flight parameters according to the strategy, there may be sudden changes due to changes in wind speed and direction, aggravation of abnormal conditions, obstacle avoidance needs, etc., so it is also necessary to make dynamic parameter adjustments in real time according to the actual environment;

[0127] Specifically, the method of dynamically adjusting parameters based on the anomaly detection information and the drone information set is as follows:

[0128] If the difference between the real-time confidence and the confidence at the previous unit time is greater than the preset confidence difference threshold, or the difference between the area covered by the real-time abnormal position and the area covered by the abnormal position at the previous unit time is greater than the preset area difference threshold, the flight speed and flight altitude are dynamically adjusted.

[0129] Specifically, the method of dynamically adjusting the flight speed and flight altitude is:

[0130] Adjust the flight speed to min(vmax, v×(1+Δc×Δs×t)); where min(·) is the minimum value function, vmax is the maximum flight speed of the drone, Δc is the difference in confidence, and Δs is the difference between the areas; that is, the larger the difference, the faster the flight speed needs to be to the abnormal location.

[0131] Adjust the flight altitude to Among them, cr is the resolution of the real-time image, and α is the preset proportional coefficient. That is, when the difference is larger, it means that the position of the drone needs to be lowered as much as possible so that the abnormal position can be captured more clearly.

[0132] Example 2

[0133] like Figure 2 As shown, the low-altitude UAV intelligent inspection system based on deep learning includes an information collection module, a detection model construction module, a strategy static adjustment module, and a strategy dynamic adjustment module; wherein each module is electrically connected;

[0134] An information collection module collects drone information sets and mission information sets in advance, sends the mission information sets to the detection model building module, and sends the drone information sets to the strategy static adjustment module;

[0135] The detection model building module builds an anomaly detection model based on the task information set and sends the anomaly detection model to the strategy static adjustment module;

[0136] The strategy static adjustment module collects the real-time flight parameter data of the UAV, builds a parameter adjustment optimization model based on the anomaly detection model, real-time flight parameter data and UAV information set, and solves and outputs the flight parameter adjustment strategy; sends the flight parameter adjustment strategy to the UAV, and collects the anomaly detection information collected during the flight of the UAV in real time, and sends the collected anomaly detection information to the strategy dynamic adjustment module;

[0137] The strategy dynamic adjustment module performs dynamic parameter adjustment based on anomaly detection information and drone information collection.

[0138] Example 3

[0139] According to another aspect of the present application, an electronic device is also provided. The electronic device may include one or more processors and one or more memories. The memory stores a computer-readable code, and when the computer-readable code is executed by one or more processors, the low-altitude UAV intelligent inspection method based on deep learning as described above can be executed.

[0140] The electronic device may include a bus, one or more CPUs, ROM, RAM, a communication port connected to a network, an input / output component, a hard disk, etc. A storage device in the electronic device, such as a ROM or a hard disk, may store the low-altitude UAV intelligent inspection method based on deep learning provided in the present application.

[0141] Furthermore, the electronic device may also include a user interface. Of course, this architecture is only exemplary, and when implementing different devices, one or more components in the electronic device may be omitted according to actual needs.

[0142] Example 4

[0143] A computer-readable storage medium according to an embodiment of the present application. Computer-readable instructions are stored on the computer-readable storage medium. When the computer-readable instructions are executed by the processor, the low-altitude unmanned aerial vehicle intelligent inspection method based on deep learning according to the embodiment of the present application described with reference to the above figures can be executed. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0144] In addition, according to the implementation of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, the non-transitory machine-readable storage medium stores machine-readable instructions, the machine-readable instructions can be run by a processor to execute instructions corresponding to the method steps provided by the present application, and when the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed.

[0145] The methods, apparatuses, and devices of the present application may be implemented in many ways. For example, the methods, apparatuses, and devices of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers recording media storing programs for executing the method according to the present application.

[0146] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0147] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0148] The above preset parameters or preset thresholds are all set by technicians in this field according to actual conditions or obtained through large-scale data simulation.

[0149] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A low-altitude UAV intelligent inspection method based on deep learning, characterized in that: The following steps are involved: Step 1: Collect drone information set and mission information set in advance; Step 2: Build an anomaly detection model based on the task information set; Step 3: Collect the real-time flight parameter data of the UAV, build a parameter adjustment optimization model based on the anomaly detection model, real-time flight parameter data and UAV information set, and solve the output flight parameter adjustment strategy; Step 4: Send flight parameter adjustment strategies to the drone and collect anomaly detection information collected during the drone flight in real time; Step 5: Dynamically adjust parameters based on anomaly detection information and drone information collection.

2. The low-altitude UAV intelligent inspection method based on deep learning according to claim 1 is characterized in that: The drone information set is collected in the following manner: According to the actual hardware configuration of the drone, various hardware parameters of the drone are counted as the drone information set; The task information set is collected in the following manner: Collect all preset inspection scenarios and corresponding inspection task types; For each type of inspection task in each inspection scenario, sample images with labeled labels and position annotations are collected in advance to form a task information set.

3. The low-altitude UAV intelligent inspection method based on deep learning according to claim 2 is characterized in that: The method of constructing anomaly detection model based on task information set is: For each inspection scenario, the corresponding inspection task type is: Read a sample image set corresponding to the inspection task type from the task information set; Divide the sample image set into training set, validation set and test set; Select the YOLO model or YOLOv3 model as the anomaly detection model; Resize all sample images to a uniform size to fit the input requirements of the anomaly detection model; Apply data augmentation techniques such as random rotation, flipping, and scaling to sample images to improve the generalization ability of the model; Preset the hyperparameter set of the anomaly detection model; The anomaly detection model uses the weighted sum of the positioning loss of bounding box regression, the classification loss of cross entropy loss, and the confidence loss of target box matching as the loss function; Use the training set to train the anomaly detection model and perform forward propagation to calculate the prediction results; Calculate the loss and optimize the model parameters of the anomaly detection model through back-propagation; Evaluate the model on the validation set, check for overfitting, and manually adjust hyperparameters until the anomaly detection model reaches the preset accuracy on the validation set; The output of the anomaly detection model includes bounding box information, confidence scores, and category labels for each detection box in the sample image; The anomaly detection model is further used on the test set to verify the model performance, and the accuracy, recall and F1 score are recorded.

4. The low-altitude UAV intelligent inspection method based on deep learning according to claim 3 is characterized in that: The method of collecting real-time flight parameter data of the drone is: According to the needs, set the actual inspection scene and inspection task type for the drone in the drone background; When controlling the drone to actually perform inspection tasks, the drone's built-in image capture equipment, sensors and navigation system are used to collect various flight parameters in real time to form real-time flight parameter data.

5. The low-altitude UAV intelligent inspection method based on deep learning according to claim 4 is characterized in that: The method of constructing a parameter adjustment optimization model based on anomaly detection model, real-time flight parameter data and drone information set, and solving the output flight parameter adjustment strategy includes the following steps: Step 31: Input the real-time image collected by the image capture device of the drone into the anomaly detection model in real time. If the confidence of the output of the anomaly detection model is greater than the preset confidence threshold, it is determined that an anomaly exists and the process goes to step 32; if the output confidence is less than or equal to the preset confidence threshold, no processing is performed; Step 32: Determine the actual geographic coordinates of the abnormal location where the abnormality occurs in the inspection scene according to the position of the bounding box output by the anomaly detection model in the real-time image and the real-time position of the drone; Step 33: Calculate the anomaly impact factor e according to the confidence output by the anomaly detection model and the actual coverage area of ​​the anomaly position; Step 34: Calculate the abnormal distance factor d according to the real-time position of the UAV and the abnormal position; Step 35: Calculate the path adjustment proportional coefficient g according to the current real-time flight speed and flight altitude of the UAV; Step 36: Adjust the flight speed according to the abnormal impact factor e; Step 37: adjusting the flight altitude according to the abnormal distance factor d; Step 38: Mark the spatial coordinates of the current position of the drone as (xc, yc, zc); adjust the path offset ΔP according to the path adjustment proportional coefficient g; the calculation formula of the path offset ΔP is: ΔP = g × (xc-xr, yc-yr); wherein xr and yr are the actual geographic horizontal coordinate and the actual geographic vertical coordinate of the abnormal position, respectively; Step 39: The abnormal impact factor e, the abnormal distance factor d and the path adjustment proportional coefficient g are used to form a flight parameter adjustment strategy.

6. The low-altitude UAV intelligent inspection method based on deep learning according to claim 5 is characterized in that: The method for determining the actual geographic coordinates of the abnormal location where the abnormality occurs in the inspection scene is: Get the anomaly bounding box data (x) from the output of the YOLO model bbox ,y bbox , w bbox ,h bbox ), where x bbox Indicates the central horizontal coordinate of the bounding box, y bbox represents the center ordinate of the bounding box, w bbox Indicates the width of the bounding box, h bbox Indicates the height of the bounding box; Use the drone's flight altitude zc and the camera's viewing angle parameters to convert the image plane coordinates into ground coordinates; Denote the viewing angle of the drone’s image capture device as θ, and calculate the conversion coefficient SC from pixels to actual distance; According to the central horizontal coordinate and the conversion coefficient, the geographic horizontal coordinate xr of the abnormal position is calculated; According to the central ordinate and the conversion coefficient, calculate the geographic ordinate yr of the abnormal position; The actual geographic coordinates of the abnormal position are expressed as (geographic horizontal coordinate, geographic vertical coordinate).

7. The low-altitude UAV intelligent inspection method based on deep learning according to claim 6 is characterized in that: The method of dynamic parameter adjustment based on anomaly detection information and drone information collection is as follows: If the difference between the real-time confidence and the confidence at the previous unit time is greater than the preset confidence difference threshold, or the difference between the area covered by the real-time abnormal position and the area covered by the abnormal position at the previous unit time is greater than the preset area difference threshold, the flight speed and flight altitude are dynamically adjusted.

8. A low-altitude UAV intelligent inspection system based on deep learning, which is used to implement the low-altitude UAV intelligent inspection method based on deep learning as described in any one of claims 1 to 7, characterized in that: It includes an information collection module, a detection model building module, a strategy static adjustment module and a strategy dynamic adjustment module; wherein each module is electrically connected; An information collection module collects drone information sets and mission information sets in advance, sends the mission information sets to the detection model building module, and sends the drone information sets to the strategy static adjustment module; The detection model building module builds an anomaly detection model based on the task information set and sends the anomaly detection model to the strategy static adjustment module; The strategy static adjustment module collects the real-time flight parameter data of the UAV, builds a parameter adjustment optimization model based on the anomaly detection model, the real-time flight parameter data and the UAV information set, solves and outputs the flight parameter adjustment strategy, sends the flight parameter adjustment strategy to the UAV, and collects the anomaly detection information collected during the flight of the UAV in real time, and sends the collected anomaly detection information to the strategy dynamic adjustment module; The strategy dynamic adjustment module performs dynamic parameter adjustment based on anomaly detection information and drone information collection.

9. An electronic device, characterized in that: include: processor and memory, wherein The memory stores a computer program that can be called by the processor; The processor executes the deep learning-based low-altitude UAV intelligent inspection method described in any one of claims 1 to 7 in the background by calling the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: A rewritable computer program is stored thereon; When the computer program runs on a computer device, the computer device executes the low-altitude UAV intelligent inspection method based on deep learning as described in any one of claims 1 to 7 in the background.

Images