Unmanned aerial vehicle inspection fault intelligent identification system and method

Abstract

The invention provides an unmanned aerial vehicle inspection fault intelligent identification system and method, and relates to the technical field of unmanned aerial vehicle intelligent inspection, and the method comprises the steps: building a dual-target path model through historical fault data, carrying out the dynamic optimization of an inspection path of an unmanned aerial vehicle through the dual-target path model according to multi-source sensing data, and obtaining an optimized path and a sensing region; performing hierarchical identification on the multi-source sensing data to obtain a state prediction value of the unmanned aerial vehicle, and constructing an intelligent path guiding function based on the optimized path, the sensing area and the state prediction value; performing back propagation correction on the intelligent path guiding function to obtain fault type features and detection precision, and constructing a feature association weight matrix of the inspection path based on the fault type features and the detection precision; according to the method and the device, path reconstruction and fault intelligent identification based on identification feedback driving can be realized, so that the accuracy of an unmanned aerial vehicle inspection result is improved.

Description

Technical Field

[0001] This application relates to the technical field of intelligent inspection of unmanned aerial vehicles (UAVs). More specifically, this application relates to a UAV inspection fault intelligent recognition system and method. Background Art

[0002] UAVs are widely used in fields such as inspection, monitoring, and emergency response. In various application scenarios, an intelligent inspection system based on UAVs can replace manual labor to complete inspection operations in large-scale, high-risk, or complex terrain areas, with advantages such as strong flexibility, high operation efficiency, and low deployment cost.

[0003] Carrying out automated UAV inspections on the netting layout in areas such as rivers, lakes, and reservoirs is one of the typical application scenarios in recent years for intelligent fishery administration, aquatic product law enforcement, and fishing ban supervision. Existing UAV intelligent inspection technologies mostly use preset paths combined with image recognition algorithms for target detection and fault recognition. However, existing methods still have strong path rigidity and lack dynamic adaptability in the actual operation process, that is: most inspection paths are based on static planning and cannot be adjusted according to real-time recognition results or target state changes, and the recognition results cannot feed back to the path generation strategy in real time, affecting the overall coordination efficiency. The fault type characteristics and recognition accuracy generated in existing UAV inspections cannot be used for adaptive correction of UAV inspection paths. Therefore, how to achieve path reconstruction and fault intelligent recognition driven by recognition feedback to improve the accuracy of UAV inspection results is a difficult problem faced by the industry. Summary of the Invention

[0004] This application provides a UAV inspection fault intelligent recognition system and method, which can achieve path reconstruction and fault intelligent recognition driven by recognition feedback to improve the accuracy of UAV inspection results.

[0005] In a first aspect, this application provides a UAV inspection fault intelligent recognition system and method. The intelligent recognition method includes the following steps:

[0006] Construct a bi-objective path model based on historical fault data of the target inspection area;

[0007] Use the UAV to collect multi-source perception data of the target inspection area. The bi-objective path model dynamically optimizes the inspection path of the UAV according to the multi-source perception data to obtain an optimized path and a perception area when the UAV conducts inspections;

[0008] Perform hierarchical recognition on the multi-source perception data through the cloud collaboration platform of the UAV to obtain a state prediction value of the UAV. Based on the optimized path, the perception area, and the state prediction value, construct an intelligent path guidance function when the UAV conducts inspections;

[0009] Perform backpropagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during UAV inspection, and construct a feature correlation weight matrix for the inspection path based on the fault type characteristics and the detection accuracy;

[0010] Identify and reconstruct the inspection path of the UAV based on the feature correlation weight matrix, and then drive the UAV to perform intelligent fault identification based on the identified and reconstructed inspection path.

[0011] In this embodiment, a non-dominated sorting genetic algorithm is used to construct a bi-objective path model based on the historical fault data of the target inspection area.

[0012] In this embodiment, the multi-source perception data includes visible light image data, infrared thermal imaging data, and three-dimensional spatial structure data, and the multi-source perception data of the target inspection area is collected by using the visible light camera, infrared thermal imager, and lidar of the UAV.

[0013] In this embodiment, the bi-objective path model dynamically optimizes the inspection path of the UAV according to the multi-source perception data to obtain the optimized path and perception area during UAV inspection, specifically including:

[0014] Determine the initial inspection path of the UAV through the multi-source perception data;

[0015] Perform risk identification and inspection efficiency optimization on the multi-source perception data to obtain dual optimization objectives;

[0016] Use the non-dominated sorting genetic algorithm to iteratively optimize the initial inspection path based on the dual optimization objectives to obtain the optimized path and perception area during UAV inspection.

[0017] In this embodiment, the multi-source perception data is hierarchically identified through the cloud collaboration platform of the UAV to obtain the state prediction value of the UAV, specifically including:

[0018] Align and fuse the multi-source perception data according to the timestamp through the cloud collaboration platform of the UAV to obtain a multi-modal input feature tensor;

[0019] Perform fine-grained identification on the multi-modal input feature tensor to obtain deep semantic features;

[0020] Based on the spatio-temporal position of the target inspection area, determine the state change trend of the UAV for the deep semantic features, and then determine the state prediction value of the UAV from the state change trend.

[0021] In this embodiment, constructing an intelligent path guidance function during UAV inspection based on the optimized path, the perception area, and the state prediction value specifically includes:

[0022] Co - encode the spatial coordinate sequence in the optimized path, the risk score matrix of the sensing area, and the state prediction value into a path guidance input vector;

[0023] Use a deep learning model based on the attention mechanism to construct a path guidance function;

[0024] Determine a path guidance vector based on the optimized path and the sensing area;

[0025] Convert the state prediction value into a state prediction vector, and then supervise and train the path guidance function through the state prediction vector and the path guidance vector to obtain an intelligent path guidance function for UAV inspection.

[0026] In this embodiment, performing backpropagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during UAV inspection specifically includes:

[0027] Obtain the fault location and fault type during UAV inspection, and predict and identify the fault location and the fault type through the intelligent path guidance function to obtain a path offset and a fault detection deviation;

[0028] Determine a penalty factor based on the path offset and the fault detection deviation;

[0029] Perform gradient backpropagation on the intelligent path guidance function through the penalty factor to obtain an intelligent path guidance function with adjusted weight parameters;

[0030] Use the intelligent path guidance function with adjusted weight parameters to correct and identify the fault location and the fault type to obtain the fault type characteristics and detection accuracy during UAV inspection.

[0031] In this embodiment, the cloud collaborative platform of the UAV is an intelligent computing platform based on deep learning and incremental training.

[0032] In this embodiment, the dual - objective path model is a path optimization model constructed by a dual - objective function, and the dual - objective function includes inspection time and abnormal area coverage rate.

[0033] In a second aspect, the present application provides a UAV inspection fault intelligent recognition system for executing a UAV inspection fault intelligent recognition method, and the intelligent recognition system includes:

[0034] A dual - objective path model construction module for constructing a dual - objective path model based on historical fault data of a target inspection area;

[0035] A path optimization module for using a drone to collect multi-source perception data of a target inspection area. The dual-objective path model dynamically optimizes the inspection path of the drone based on the multi-source perception data to obtain an optimized path and a perception area during drone inspection.

[0036] A path guidance module for hierarchically identifying the multi-source perception data through the cloud collaborative platform of the drone to obtain a state prediction value of the drone, and constructing an intelligent path guidance function during drone inspection based on the optimized path, the perception area, and the state prediction value.

[0037] A weight construction module for performing backpropagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during drone inspection, and constructing a feature association weight matrix for the inspection path based on the fault type characteristics and the detection accuracy.

[0038] A path reconstruction execution module for identifying and reconstructing the inspection path of the drone based on the feature association weight matrix, and then driving the drone to perform intelligent fault identification based on the identified and reconstructed inspection path.

[0039] The technical solution provided by the disclosed embodiments of the present application has the following beneficial effects:

[0040] Construct a dual-objective path model based on the historical fault data of the target inspection area; use a drone to collect multi-source perception data of the target inspection area. The dual-objective path model dynamically optimizes the inspection path of the drone based on the multi-source perception data to obtain an optimized path and a perception area during drone inspection; hierarchically identify the multi-source perception data through the cloud collaborative platform of the drone to obtain a state prediction value of the drone, and construct an intelligent path guidance function during drone inspection based on the optimized path, the perception area, and the state prediction value; perform backpropagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during drone inspection, and construct a feature association weight matrix for the inspection path based on the fault type characteristics and the detection accuracy; identify and reconstruct the inspection path of the drone based on the feature association weight matrix, and then drive the drone to perform intelligent fault identification based on the identified and reconstructed inspection path.

[0041] It can be seen that in this application, path reconstruction and intelligent fault identification driven by recognition feedback can be realized. First, by adopting the non-dominated sorting genetic algorithm and constructing a bi-objective path model based on the historical fault data of the target inspection area, the coverage efficiency of the inspection path and the attention weight of high-risk areas can be taken into account simultaneously, realizing multi-objective intelligent modeling of the initial inspection path and providing a structural prior reference for subsequent path dynamic optimization. Second, using the drone to collect multi-source perception data of the target area, including visible light images, infrared thermal imaging, and spatial point cloud and other information, and combining the bi-objective path model to realize dynamic optimization processing of the path, which can not only adapt to environmental changes in real time but also ensure multi-dimensional coverage of key areas, thus improving the integrity of inspection data and the dynamic response ability of the path. Then, through the cloud collaboration platform of the drone, hierarchical identification of the multi-source perception data is carried out, and an intelligent path guidance function is constructed by combining the optimized path, the perception area, and the state prediction value, enabling the path planning process to have state perception ability, realizing adaptive path scheduling under recognition guidance, and enhancing the correlation between the recognition area and the path node. Then, by obtaining the fault detection deviation and the path offset, performing backpropagation correction on the intelligent path guidance function, and constructing a feature correlation weight matrix for the inspection path by combining the detection accuracy and the fault type, not only can the path guidance strategy be feedback optimized, but also the mapping relationship between the recognition risk and the path priority can be quantified, constructing a recognition-driven path control model. Finally, based on the feature correlation weight matrix, the inspection path is recognized and reconstructed, and the drone is driven to perform the fault detection task according to the recognized and reconstructed path, thus realizing the efficient coupling of path planning and recognition decision-making, and significantly improving the fault detection accuracy and path adaptability of the drone in a dynamic environment.

[0042] In summary, the technical solution adopted in this application can realize path reconstruction and intelligent fault identification driven by recognition feedback to improve the accuracy of drone inspection results. Brief Description of the Drawings

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0044] Figure 1 is a flowchart of an intelligent drone inspection fault identification method provided by the present application;

[0045] Figure 2 is an exemplary flowchart for determining the optimized path and perception area during drone inspection provided by the present application;

[0046] Figure 3 It is an exemplary flowchart for determining the state prediction value of a drone provided according to this application;

[0047] Figure 4 It is a module structure diagram of an intelligent recognition system provided according to this application. Detailed implementation manners

[0048] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0049] The embodiments of this application provide an intelligent recognition system and method for drone inspection faults. The core is to construct a dual-objective path model based on the historical fault data of the target inspection area; use the drone to collect multi-source perception data of the target inspection area, and the dual-objective path model dynamically optimizes the inspection path of the drone according to the multi-source perception data to obtain the optimized path and perception area when the drone conducts inspections; perform hierarchical recognition on the multi-source perception data through the cloud collaboration platform of the drone to obtain the state prediction value of the drone, and construct an intelligent path guidance function when the drone conducts inspections based on the optimized path, the perception area, and the state prediction value; perform backpropagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy when the drone conducts inspections, and construct a feature correlation weight matrix for the inspection path based on the fault type characteristics and the detection accuracy; perform recognition and reconstruction on the inspection path of the drone based on the feature correlation weight matrix, and then drive the drone to perform intelligent fault recognition based on the recognized and reconstructed inspection path.

[0050] Embodiment 1. To better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners. Refer to Figure 1 As shown, this figure is an exemplary flowchart of an intelligent recognition method for drone inspection faults shown in this embodiment of this application. The intelligent recognition method includes the following steps:

[0051] In step S1, construct a dual-objective path model based on the historical fault data of the target inspection area.

[0052] In specific implementation, a non-dominated sorting genetic algorithm is used to construct a bi-objective path model based on the historical fault data of the target inspection area, that is: First, the historical fault data of the target inspection area can be obtained through the database of the target inspection area. The historical fault data includes: geographical location information, time tags, fault types, and fault degrees. Then, a fault density map is constructed based on the historical fault data through a visualization algorithm. The fault density map is used to reflect the spatial distribution characteristics of the target inspection area. And the target inspection area is discretized into several inspection nodes through a random forest algorithm, and a weighted directed graph is jointly constructed with the geographical distance between the inspection nodes and the historical fault density of the area to which the nodes belong. Then, minimizing the total time-consuming of the inspection path is used as the first objective function, and the coverage rate of the historical high-fault density area is used as the second objective function to obtain a bi-objective function. The weighted directed graph is optimized for the path through the non-dominated sorting genetic algorithm and the bi-objective function. The optimized graph model is used as the bi-objective path model.

[0053] It should be noted that the bi-objective path model in this embodiment is a path optimization model constructed by a bi-objective function. The bi-objective function includes inspection time and the coverage rate of abnormal areas, which is beneficial to improving the execution efficiency of the inspection task and the coverage ability of key potential hazard areas. Among them, the relationship between inspection nodes is represented by constructing a weighted directed graph, and the historical fault density is used as a quantitative index for the coverage rate of abnormal areas, so that the model can adapt to the distribution characteristics of the key areas of the inspection task. In addition, the non-dominated sorting genetic algorithm has strong global optimization ability and the characteristics of adapting to multi-objective optimization problems, and can effectively balance between various inspection requirements, so as to output an optimal inspection path set with practical guiding significance.

[0054] In step S2, multi-source perception data of the target inspection area is collected by the UAV. The bi-objective path model dynamically optimizes the inspection path of the UAV according to the multi-source perception data to obtain the optimized path and the perception area during the UAV inspection.

[0055] It should be noted that in this application, the multi-source perception data includes visible light image data, infrared thermal imaging data, and three-dimensional spatial structure data. The multi-source perception data of the target inspection area is collected by using the visible light camera, infrared thermal imager, and lidar of the unmanned aerial vehicle (UAV). Among them, the visible light image data is used to identify visible features such as appearance damage and obstacles, the infrared thermal imaging data is used to identify thermal features such as temperature anomalies and hot spot concentration, and the three-dimensional spatial structure data is used to identify geometric features such as structural deformation, distance measurement, and spatial layout. The high-dimensional features of the target inspection area can be constructed through the multi-source perception data, enhancing the system's comprehensive perception ability of abnormal risks in complex environments and providing high-quality data support for subsequent path optimization, risk modeling, and decision-making analysis. In addition, the multi-source perception data of the target inspection area is collected by using the visible light camera, infrared thermal imager, and lidar of the UAV. The visible light camera is used to collect visible light image data, the infrared thermal imager is used to collect infrared thermal imaging data, and the lidar is used to collect three-dimensional spatial structure data, which is beneficial to meeting the requirements of carrying out multi-scene inspection tasks in different types of target areas.

[0056] In specific implementation, the visible light image data, infrared thermal imaging data, and lidar point cloud data of the target inspection area are obtained by using a UAV equipped with a visible light camera, an infrared thermal imager, and a lidar. Among them, the resolution of the visible light camera can be 1080p, which is used to provide surface structure and texture information of the target area. The infrared thermal imager collects thermal image data of the target area through the induction of thermal radiation, and can identify temperature anomalies or heating equipment. The lidar collects three-dimensional spatial structure data based on the time-of-flight principle, obtains the three-dimensional coordinates and reflection intensity information of the surface points of the target area, and finally maps the collected visible light image data, infrared thermal imaging data, and three-dimensional spatial structure data into multi-source fusion perception data through an existing projection matrix.

[0057] Preferably, in this embodiment, refer to Figure 2 As shown in the figure, this figure is an exemplary flowchart for determining the optimized path and perception area during UAV inspection provided by this application. In this embodiment, the dual-objective path model dynamically optimizes the inspection path of the UAV based on the multi-source perception data, and the specific steps for obtaining the optimized path and perception area during UAV inspection can be implemented as follows:

[0058] First, in step S21, the initial inspection path of the UAV is determined through the multi-source perception data;

[0059] Then, in step S22, risk identification and inspection efficiency optimization are performed on the multi-source perception data to obtain dual optimization objectives;

[0060] Finally, in step S23, the non-dominated sorting genetic algorithm is used to iteratively optimize the initial inspection path based on the dual optimization objectives to obtain the optimized path and the sensing area during the UAV inspection.

[0061] In specific implementation, first, the multi-source sensing data is spatially rasterized, and the target inspection area is divided into multiple sensing units. Among them, each sensing unit corresponds to a set of sensing data including visible light images, infrared thermal imaging images, and three-dimensional structure information. The initial inspection path of the UAV can be obtained by using the area coverage algorithm to identify the spatial distribution of all sensing units. Then, through time synchronization and spatial alignment processing of different modality input data, a unified input tensor is formed. Then, the pre-trained convolutional neural network (CNN) and long short-term memory network (LSTM) are used to fuse and extract the spatio-temporal feature vectors, and they are input into the classification and discrimination layer to output the abnormal category probability of each sensing unit. At the same time, according to indicators such as the voyage length, inspection time, and area coverage rate in the inspection flight path, an inspection efficiency function is constructed to quantify the path execution efficiency. Thus, the inspection efficiency function and the abnormal category probability are used as the dual optimization objectives. Finally, the abnormal category probability and the inspection efficiency function are used as the constraint parameters of the non-dominated sorting genetic algorithm. Thus, the loss function of the non-dominated sorting genetic algorithm with the set constraint parameters is used as the dual-objective optimization function. Among them, in the optimization process, the coding method uses the access order of the sensing units as the gene structure. Multiple path candidate solutions are generated through crossover and mutation operations. The risk identification value and the inspection efficiency value of each solution are calculated, and non-dominated sorting and population update are performed according to the Pareto optimality principle to obtain the optimized path and the sensing area during the UAV inspection. It should be noted that the crossover operation can achieve path recombination by exchanging the gene segments of two parent paths. The mutation operation breaks the local optimal trap by adjusting the access order, improves the solution space exploration ability, and the optimization result is controlled by the elitist retention strategy to retain high-quality solutions to ensure evolutionary stability.

[0062] It should be noted that the dual-objective path model avoids the problem of missing high-risk areas that may be caused by simply taking the shortest path or the least time as the goal by integrating two optimization dimensions of risk identification and path efficiency. Among them, the dual-objective path optimization implemented by the non-dominated sorting genetic algorithm has stronger diversity preservation ability and higher global convergence performance, and can adaptively adjust the flight path in a complex inspection environment. In addition, the optimized path contains flight order information, which is convenient for realizing task adaptive allocation.

[0063] In step S3, the multi-source sensing data is hierarchically identified through the cloud collaboration platform of the UAV to obtain the state prediction value of the UAV, and an intelligent path guidance function during the UAV inspection is constructed based on the optimized path, the sensing area, and the state prediction value.

[0064] It should be noted that in this application, the cloud collaborative platform of the unmanned aerial vehicle (UAV) is an intelligent computing platform based on deep learning and incremental training, which has a distribution mechanism based on knowledge distillation and the ability to fuse recognition results. The cloud collaborative platform of the UAV is a heterogeneous fusion system integrating data processing, model training, and task scheduling. The cloud collaborative platform has the ability to efficiently process multi-source perception data uploaded by the UAV during flight, and can realize remote inference deployment and online iterative optimization of the model. Among them, the cloud collaborative platform performs incremental training on various perception models, that is, on the basis of the original training model, small-batch rapid learning and updating are performed according to newly collected data, which can avoid full-scale retraining and improve the adaptability and update efficiency of the model. In addition, the cloud collaborative platform includes a spatio-temporal synchronization modeling algorithm, a multi-modal deep fusion recognition network, and a spatio-temporal evolution modeling network. Based on the cloud collaborative platform, resource scheduling and model fusion can be performed on the task states of multiple UAVs to ensure data consistency under multi-aircraft collaborative operations.

[0065] Preferably, in this embodiment, referring to Figure 3 As shown, this figure is an exemplary flowchart for determining the state prediction value of the UAV provided by this application. In this embodiment, the multi-source perception data is hierarchically recognized through the cloud collaborative platform of the UAV, and the state prediction value of the UAV can be specifically realized by the following steps:

[0066] First, in step S31, the multi-source perception data is aligned and fused according to the time stamp through the cloud collaborative platform of the UAV to obtain a multi-modal input feature tensor;

[0067] Then, in step S32, fine-grained recognition is performed on the multi-modal input feature tensor to obtain deep semantic features;

[0068] Finally, in step S33, based on the spatio-temporal position of the target inspection area, the state change trend of the UAV is determined for the deep semantic features, and then the state prediction value of the UAV is determined from the state change trend.

[0069] In specific implementation, first, the cloud collaborative platform aligns and fuses the multi-source perception data according to timestamps through a spatio-temporal synchronization modeling algorithm to obtain a multi-modal input feature tensor, which includes spatio-temporal sequence information, spatial structure information, and thermal anomaly response information. Then, the multi-modal deep fusion recognition network of the cloud collaborative platform performs fine-grained recognition on the multi-modal input feature tensor, and uses the output features of the fine-grained recognition as deep semantic features. Among them, the multi-modal deep fusion recognition network includes a convolutional neural network, a point cloud neural network, and a multi-head self-attention mechanism, which can extract content such as structural defect features, thermal anomaly response features, and spatial geometric deformation features. Finally, based on the spatio-temporal evolution modeling network of the cloud collaborative platform, the deep semantic features are modeled in combination with the spatio-temporal position of the target inspection area to determine the state change trend of the UAV under specific spatio-temporal conditions. Furthermore, the task delay parameter in the state change trend is analyzed through the spatio-temporal evolution modeling network, and this task delay parameter is used as the state prediction value of the UAV. Among them, this spatio-temporal evolution modeling network can be constructed based on the Transformer architecture, which is beneficial to maintaining spatial consistency and temporal continuity, so as to output the state change trend of the UAV.

[0070] It should be noted that the state prediction value in this application is based on the dynamic evolution characteristics during the task execution process. Through the spatio-temporal modeling of deep semantic features, it outputs task delay data related to a future time period, which can be used to quantitatively evaluate the operation ability and potential risks of the UAV in subsequent inspection tasks. That is, according to the predicted task delay, the task priority or perception strategy can be dynamically adjusted to achieve efficient resource allocation.

[0071] In this embodiment, the intelligent path guidance function for UAV inspection is specifically constructed based on the optimized path, the perception area, and the state prediction value in the following way, that is:

[0072] The spatial coordinate sequence in the optimized path, the risk score matrix of the perception area, and the state prediction value are jointly encoded into a path guidance input vector;

[0073] A path guidance function is constructed using a deep learning model based on the attention mechanism;

[0074] The path guidance vector is determined through the optimized path and the perception area;

[0075] The state prediction value is converted into a state prediction vector, and then the path guidance function is supervised and trained through the state prediction vector and the path guidance vector to obtain the intelligent path guidance function for UAV inspection.

[0076] In specific implementation, first, extract the spatial coordinate sequence in the optimized path through the dual-objective path model, and use the dual-objective path model to construct a risk scoring matrix for the perception area. After standardizing the state prediction value, the spatial coordinate sequence, and the risk scoring matrix, convert them into a path guidance input vector through one-hot encoding. Secondly, preset an attention mechanism based on the computational complexity of the standardization process, and construct a path guidance function based on the Transformer network. Then, perform secondary training and learning on the Transformer network through the optimized path and the perception area. Furthermore, the path guidance vector is obtained by the secondary-trained Transformer network through real-time inspection path prediction of the unmanned aerial vehicle (UAV). Finally, input the state prediction vector and map it to the same feature dimension as the path guidance vector, and then use cross-entropy as the loss function to supervise and train the path guidance function, and optimize the network weights through the backpropagation algorithm during the supervision and training process, so as to obtain the intelligent path guidance function for UAV inspection.

[0077] It should be noted that the intelligent path guidance function constructed in this embodiment has the ability of adaptive dynamic adjustment, which can respond to path changes in real time during operation, facilitating the implementation of optimal decision-making guidance; the intelligent path guidance function has the ability to fuse high-dimensional heterogeneous inputs, can effectively extract the deep semantic associations between data from different sources, and significantly improve the rationality of UAV inspection path selection; in addition, by introducing the attention mechanism, the intelligent path guidance function can focus on the key areas and high-risk targets in the current inspection task, improving the overall path guidance accuracy; the intelligent path guidance function can be deployed on the cloud collaborative platform of the UAV, and real-time path optimization is carried out through the wireless communication function of the UAV.

[0078] In step S4, perform backpropagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during UAV inspection, and construct a feature correlation weight matrix for the inspection path based on the fault type characteristics and the detection accuracy.

[0079] In this embodiment, performing backpropagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during UAV inspection can be specifically carried out in the following manner, that is:

[0080] Obtain the fault location and fault type during UAV inspection, and perform prediction and identification on the fault location and the fault type through the intelligent path guidance function to obtain the path offset and the fault detection deviation.

[0081] Determine the penalty factor based on the path offset and the fault detection deviation.

[0082] Perform gradient backpropagation on the intelligent path guidance function through the penalty factor to obtain the intelligent path guidance function with adjusted weight parameters;

[0083] Use the intelligent path guidance function with adjusted weight parameters to correct and identify the fault location and the fault type, and obtain the fault type characteristics and detection accuracy during the UAV inspection.

[0084] Specifically, when implementing, first, the multi-modal data transmitted through the wireless communication function of the UAV is divided by the recognition function pre-trained based on manual annotation to obtain the fault location and the fault type. The intelligent path guidance function is used to predict and output the fault location and the fault type. Then, the spatial offset distance between the predicted location and the actual fault location is used as the path offset, and the class difference between the predicted fault label and the actual label is used as the fault detection deviation. Among them, the class difference can be measured by categorical cross-entropy. Secondly, the weighted sum value of the path offset and the fault detection deviation is used as the penalty factor. Preferably, the weights of the weighted sum can be based on the validation set and adopted the Bayesian optimization method to traverse different weight combinations and evaluate indicators such as the inspection accuracy rate and the average path offset of the model. Finally, the optimal weight is selected. In other embodiments, the weight can also be determined by other methods, which are not limited here. Then, the penalty factor is substituted into the path guidance function as a constant term to obtain a new expression, and the parameters in the new expression are updated by gradient backpropagation through the backpropagation algorithm. Through the above update process, the intelligent path guidance function with adjusted weight parameters can be obtained. Preferably, the gradient backpropagation update process can use the gradient descent optimizer to adjust the weights of the multi-source input feature channels and the attention mechanism weights, which is beneficial to strengthening the model's perception ability of the fault spatial distribution and type difference. In other embodiments, other optimizers with the same adjustment effect can also be used for adjustment, which are not limited here. Finally, use the intelligent path guidance function after weight adjustment to predict and identify the fault location and the fault type obtained by the UAV, so as to output the fault type characteristics and detection accuracy during the UAV inspection, extract the fault type characteristics and calculate the detection accuracy. Among them, the detection accuracy can be evaluated by the recall rate.

[0085] It should be noted that the intelligent path guidance function in this embodiment can perform path planning and dynamic scheduling for the UAV; by establishing a penalty factor mechanism through the path offset and the fault detection deviation, it can effectively identify the error sources in the spatial positioning and type determination of the model, so as to achieve dynamic correction during the training process, enhance the adaptability of the fault characteristics, and facilitate driving the UAV to perform intelligent fault inspection; in addition, by performing real-time update on the intelligent path guidance function through the backpropagation correction process, the parameters can be adjusted in combination with the real-time perception results, so that the guidance function continuously adapts to environmental changes and diverse fault forms, and ensures the recognition accuracy of the UAV during the long-term and multi-task inspection process.

[0086] In this embodiment, the feature correlation weight matrix of the inspection path is specifically constructed based on the fault type feature and the detection accuracy in the following manner, that is:

[0087] Determine the risk level score according to the fault type feature;

[0088] Determine the detection confidence through the detection accuracy and the risk level score;

[0089] Construct a weight matrix based on the node features of the inspection path, and then optimize the association of the weight matrix by the detection confidence to obtain the feature correlation weight matrix of the inspection path.

[0090] Specifically, first, the fault type feature includes the fault category, fault location, and fault degree. Encode multiple dimensions of the fault type feature into a feature vector, and then perform a regression analysis on the risk level based on historical experience and the feature vector to obtain the risk level score. Then, multiply the detection accuracy by the reciprocal of the risk level score, and use the obtained product as the detection confidence. Among them, the detection confidence represents the reliability of the fault identification result. The higher the detection confidence value, the stronger the reliability of the fault identification result, and it can be given priority in subsequent decisions such as path scheduling, task planning, and risk warning. The lower the detection confidence, corresponding countermeasures such as downgrading processing, alternative paths, and enhanced inspections can be taken. Finally, perform feature encoding on the nodes where faults are identified in the inspection path of the UAV to obtain node features. The node features include the node location and the fault type. The node location and the fault type can be converted into a node feature vector through one-hot encoding, and then the matrix formed by arranging the node feature vectors row by row is used as the weight matrix. Then, perform weight association on each row of the weight matrix through the detection confidence. The detection confidence only assigns weights to the fault types, that is, in the weight matrix, the fault types corresponding to each row in the weight matrix are weighted by the detection confidence. The matrix obtained after weight assignment is used as the feature correlation weight matrix of the inspection path, which can be used for dynamic programming in the path reconstruction stage and can also be used as the adjacency weight matrix for input to the graph neural network.

[0091] It should be noted that the feature correlation weight matrix in this application refers to an expression matrix of the association strength between path nodes constructed through feature similarity calculation and confidence weighted optimization based on multi-dimensional features such as the spatial information, task attributes, fault types, and detection confidence of each path node in the UAV inspection path, which can be used to quantify the risk propagation ability between different nodes in the inspection task; in addition, by optimizing the weight distribution between nodes through the detection confidence, nodes with high risk and high recognition accuracy can be preferentially considered during the path planning process, thereby improving the reliability and efficiency of the overall inspection strategy.

[0092] In step S5, the inspection path of the UAV is identified and reconstructed based on the feature correlation weight matrix, and then the UAV is driven to perform intelligent fault identification based on the identified and reconstructed inspection path.

[0093] In this embodiment, the inspection path of the UAV is identified and reconstructed based on the feature correlation weight matrix, and then the UAV is driven to perform intelligent fault identification based on the identified and reconstructed inspection path. Specifically, the following method can be adopted, that is:

[0094] Map the feature correlation weight matrix to the node features of the inspection path, and then determine the reconstructed path sequence of the inspection path based on the particle swarm optimization algorithm;

[0095] The cloud collaborative platform of the UAV drives the UAV to perform intelligent fault identification based on the identified and reconstructed inspection path according to the reconstructed path sequence.

[0096] When specifically implemented, first map the feature correlation weight matrix to the node feature space of the inspection path, realize the association and fusion of the weight matrix and node features by constructing a weight mapping function, obtain the weighted feature representation, and based on this weighted feature representation, use the particle swarm optimization algorithm to perform global search and local adjustment on the inspection path, and then use the optimal access sequence of the obtained path nodes as the reconstructed path sequence of the inspection path; among them, the fitness function of the particle swarm optimization algorithm comprehensively considers the path length, node risk weight, and detection confidence to ensure that the path not only meets the inspection efficiency but also preferentially covers high-risk areas; in actual implementation, multiple particles can be initialized to represent different path sequences, and the speed and position of the particles can be iteratively updated to gradually approach the optimal path solution until the convergence condition is met or the preset number of iterations is reached; then, the cloud collaborative platform of the UAV dynamically adjusts the fault identification path to the UAV end through the reconstructed path sequence, and then drives the UAV to perform intelligent fault identification based on the dynamically adjusted fault identification path through wireless drive technology.

[0097] It should be noted that the inspection path recognition and reconstruction in this application refers to re - sorting and restructuring the original path node sequence through an optimization algorithm based on the risk correlation degree and detection confidence information between nodes provided by the feature correlation weight matrix, so as to generate an inspection path sequence that conforms to the current perception ability, which is beneficial to improving the path decision - making ability of the UAV in complex inspection scenarios, making the path planning result have structural rationality. Among them, the reconstructed path sequence can be synchronously transmitted to the cloud collaborative platform, and combined with the incremental learning mechanism to optimize the recognition model, so that the fault recognition strategy at the UAV end can be updated in real time and dynamically adapt to the changing task objectives, improving the overall intelligence level of the system.

[0098] It can be seen that in this application, path reconstruction and intelligent fault recognition driven by recognition feedback can be realized. First, by adopting the non - dominated sorting genetic algorithm and constructing a two - objective path model based on the historical fault data of the target inspection area, the coverage efficiency of the inspection path and the attention weight of high - risk areas can be taken into account at the same time, realizing multi - objective intelligent modeling of the initial inspection path and providing a structural prior reference for subsequent path dynamic optimization. Second, using the UAV to collect multi - source perception data of the target area, including visible light images, infrared thermal imaging, and spatial point cloud and other information, and combining with the two - objective path model to realize dynamic optimization processing of the path, which can not only adapt to environmental changes in real time, but also ensure multi - dimensional coverage of key areas, thus improving the integrity of inspection data and the dynamic response ability of the path. Then, through the cloud collaborative platform of the UAV, hierarchical recognition of the multi - source perception data is carried out, and an intelligent path guidance function is constructed by combining the optimized path, perception area, and state prediction value, enabling the path planning process to have state perception ability, realizing adaptive path scheduling under recognition guidance, and improving the correlation between the recognition area and path nodes. Then, by obtaining the fault detection deviation and path offset, performing backpropagation correction on the intelligent path guidance function, and constructing a feature correlation weight matrix of the inspection path by combining detection accuracy and fault type, not only can the path guidance strategy be feedback - optimized, but also the mapping relationship between recognition risk and path priority can be quantified, constructing a recognition - driven path control model. Finally, based on the feature correlation weight matrix, the inspection path is recognized and reconstructed, and the UAV is driven to perform fault detection tasks according to the recognized and reconstructed path, thus realizing the efficient coupling of path planning and recognition decision - making, and significantly improving the fault detection accuracy and path adaptability of the UAV in a dynamic environment.

[0099] In summary, the technical solution adopted in this application can realize path reconstruction and intelligent fault recognition driven by recognition feedback to improve the accuracy of UAV inspection results.

[0100] Embodiment 2. This application provides an intelligent UAV inspection fault recognition system. Refer to Figure 4As shown, this figure is a module structure diagram of the intelligent recognition system according to this embodiment of the present application. The intelligent recognition system includes:

[0101] A dual-target path model construction module 100, configured to construct a dual-target path model based on historical fault data of a target inspection area;

[0102] A path optimization module 200, configured to use a drone to collect multi-source perception data of the target inspection area. The dual-target path model dynamically optimizes the inspection path of the drone according to the multi-source perception data to obtain an optimized path and a perception area when the drone conducts inspections;

[0103] A path guidance module 300, configured to perform hierarchical identification on the multi-source perception data through a cloud collaboration platform of the drone to obtain a state prediction value of the drone, and construct an intelligent path guidance function when the drone conducts inspections based on the optimized path, the perception area, and the state prediction value;

[0104] A weight construction module 400, configured to perform backpropagation correction on the intelligent path guidance function to obtain a fault type feature and a detection accuracy when the drone conducts inspections, and construct a feature association weight matrix of the inspection path based on the fault type feature and the detection accuracy;

[0105] A path reconstruction execution module 500, configured to perform identification and reconstruction on the inspection path of the drone based on the feature association weight matrix, and further drive the drone to perform intelligent fault identification based on the identified and reconstructed inspection path.

[0106] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each process and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions, and the combination of processes and / or blocks in the flowcharts and / or block diagrams can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0107] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disk memories, tape memories, or any other medium that can be used to carry or store data and is computer-readable.

[0108] It should also be noted that the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent in such a process, method, commodity or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, commodity or device including the element.

Claims

1. An intelligent recognition method for drone inspection faults, characterized in that, The intelligent recognition method includes the following steps: Construct a bi-objective path model based on the historical fault data of the target inspection area; Use the drone to collect multi-source perception data of the target inspection area. The bi-objective path model dynamically optimizes the inspection path of the drone according to the multi-source perception data to obtain the optimized path and perception area during the drone inspection; Perform hierarchical recognition on the multi-source perception data through the cloud collaborative platform of the drone to obtain the state prediction value of the drone. Construct an intelligent path guidance function during the drone inspection based on the optimized path, the perception area, and the state prediction value; Perform backpropagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during the drone inspection. Construct a feature correlation weight matrix of the inspection path based on the fault type characteristics and the detection accuracy; Perform recognition and reconstruction on the inspection path of the drone based on the feature correlation weight matrix, and then drive the drone to perform intelligent fault recognition based on the recognized and reconstructed inspection path.

2. The intelligent fault identification method for UAV inspection according to claim 1, wherein Use the non-dominated sorting genetic algorithm to construct a bi-objective path model based on the historical fault data of the target inspection area.

3. The intelligent fault identification method for UAV inspection as described in claim 1, characterized in that, The multi-source perception data includes visible light image data, infrared thermal imaging data, and three-dimensional space structure data. Use the visible light camera, infrared thermal imager, and lidar of the drone to collect the multi-source perception data of the target inspection area.

4. The intelligent fault identification method for drone inspection as described in claim 1, wherein, The bi-objective path model dynamically optimizes the inspection path of the drone according to the multi-source perception data to obtain the optimized path and perception area during the drone inspection, which specifically includes: Determine the initial inspection path of the drone through the multi-source perception data; Perform risk identification and inspection efficiency optimization on the multi-source perception data to obtain two optimization objectives; Use the non-dominated sorting genetic algorithm to iteratively optimize the initial inspection path based on the two optimization objectives to obtain the optimized path and perception area during the drone inspection.

5. The intelligent fault identification method for UAV inspection according to claim 1, characterized in that, Perform hierarchical recognition on the multi-source perception data through the cloud collaborative platform of the drone to obtain the state prediction value of the drone, which specifically includes: Align and fuse the multi-source perception data according to the time stamp through the cloud collaborative platform of the drone to obtain a multi-modal input feature tensor; Perform fine-grained recognition on the multi-modal input feature tensor to obtain deep semantic features; Determine the state change trend of the drone based on the deep semantic features according to the spatio-temporal position of the target inspection area, and then determine the state prediction value of the drone from the state change trend.

6. The intelligent fault identification method for UAV inspection according to claim 1, characterized in that, Construct an intelligent path guidance function during the drone inspection based on the optimized path, the perception area, and the state prediction value, which specifically includes: Jointly encode the spatial coordinate sequence in the optimized path, the risk score matrix of the perception area, and the state prediction value into a path guidance input vector; Use a deep learning model based on the attention mechanism to construct a path guidance function; Determine a path guidance vector through the optimized path and the perception area; Convert the state prediction value into a state prediction vector, and then perform supervised training on the path guidance function through the state prediction vector and the path guidance vector to obtain an intelligent path guidance function during the drone inspection.

7. The intelligent fault identification method for drone inspection as described in claim 1, wherein Performing backpropagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during UAV inspection specifically includes: Obtaining the fault location and fault type during UAV inspection, and predicting and identifying the fault location and the fault type through the intelligent path guidance function to obtain the path offset and fault detection deviation; Determining a penalty factor based on the path offset and the fault detection deviation; Performing gradient backpropagation on the intelligent path guidance function through the penalty factor to obtain the intelligent path guidance function with adjusted weight parameters; Using the intelligent path guidance function with adjusted weight parameters to perform correction and identification on the fault location and the fault type to obtain the fault type characteristics and detection accuracy during UAV inspection.

8. The intelligent fault identification method for UAV inspection as claimed in claim 1, wherein, The cloud collaborative platform of the UAV is an intelligent computing platform based on deep learning and incremental training.

9. The intelligent fault identification method for drone inspection according to claim 1, wherein, The dual-objective path model is a path optimization model constructed by a dual-objective function, and the dual-objective function includes inspection time and abnormal area coverage rate.

10. A UAV inspection fault intelligent identification system for performing a UAV inspection fault intelligent identification method according to any one of claims 1 to 9, characterized in that, The intelligent recognition system includes: A dual-objective path model construction module for constructing a dual-objective path model based on the historical fault data of the target inspection area; A path optimization module for using the UAV to collect multi-source perception data of the target inspection area, and the dual-objective path model dynamically optimizing the inspection path of the UAV according to the multi-source perception data to obtain the optimized path and perception area during UAV inspection; A path guidance module for hierarchically identifying the multi-source perception data through the cloud collaborative platform of the UAV to obtain the state prediction value of the UAV, and constructing an intelligent path guidance function during UAV inspection based on the optimized path, the perception area, and the state prediction value; A weight construction module for performing backpropagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during UAV inspection, and constructing a feature correlation weight matrix of the inspection path based on the fault type characteristics and the detection accuracy; A path reconstruction execution module for identifying and reconstructing the inspection path of the UAV based on the feature correlation weight matrix, and then driving the UAV to perform intelligent fault identification based on the identified and reconstructed inspection path.

Images