An intelligent identification system and method for drone inspection faults

Abstract

The present application provides a system and method for intelligent fault identification of unmanned aerial vehicle inspection, which relates to the technical field of intelligent unmanned aerial vehicle inspection. A dual-target path model is constructed through historical fault data. The inspection path of the unmanned aerial vehicle is dynamically optimized according to multi-source perception data through the dual-target path model to obtain an optimized path and perception area; the multi-source perception data is layered and identified to obtain a state prediction value of the unmanned aerial vehicle, and an intelligent path guidance function is constructed based on the optimized path, perception area and state prediction value; the intelligent path guidance function is back-propagated and corrected to obtain fault type characteristics and detection accuracy, and a feature association weight matrix of the inspection path is constructed based on the fault type characteristics and detection accuracy; and the unmanned aerial vehicle is driven to perform intelligent fault identification on the inspection path after identification and reconstruction based on the feature association weight matrix. The present application can realize path reconstruction and intelligent fault identification driven by identification feedback, so as to improve the accuracy of the unmanned aerial vehicle inspection results.

Description

Technical Field

[0001] The present application relates to the technical field of drone intelligent inspection, and more specifically, to a drone inspection fault intelligent identification system and method. Background Art

[0002] Drones are widely used in inspection, monitoring, and emergency response. In various application scenarios, drone-based intelligent inspection systems can replace manual inspections in large-scale, high-risk, or complex terrain areas. They have the advantages of strong flexibility, high operational efficiency, and low deployment costs.

[0003] Automated drone inspections of fishing nets in rivers, lakes, reservoirs, and other areas have become a prominent application in recent years for smart fishery administration, aquatic law enforcement, and fishing bans. Existing intelligent drone inspection technologies often use preset paths combined with image recognition algorithms for target detection and fault identification. However, these methods suffer from strong path rigidity and a lack of dynamic adaptability in practice. Most inspection paths are based on static planning and cannot be adjusted based on real-time recognition results or changes in target status. Furthermore, recognition results cannot be fed back into the path generation strategy in real time, impacting overall collaborative efficiency. The fault type characteristics and recognition accuracy generated during existing drone inspections preclude adaptive correction of drone inspection paths. Therefore, achieving path reconstruction and intelligent fault identification driven by recognition feedback to improve the accuracy of drone inspection results remains a challenge facing the industry. Summary of the Invention

[0004] The present application provides a system and method for intelligent fault identification in drone inspections, which can realize path reconstruction and intelligent fault identification based on identification feedback drive, so as to improve the accuracy of drone inspection results.

[0005] In a first aspect, the present application provides a system and method for intelligently identifying faults during drone inspections. The intelligent identification method comprises the following steps:

[0006] A dual-target path model is constructed based on historical fault data in the target inspection area;

[0007] The dual-target path model dynamically optimizes the inspection path of the drone based on the multi-source perception data to obtain the optimized path and perception area of ​​the drone during inspection.

[0008] The multi-source perception data is hierarchically identified through the UAV's cloud collaborative platform to obtain a state prediction value of the UAV, and an intelligent path guidance function for the UAV inspection is constructed based on the optimized path, the perception area, and the state prediction value;

[0009] Performing back-propagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during the drone inspection, and constructing a feature association weight matrix of the inspection path based on the fault type characteristics and the detection accuracy;

[0010] The inspection path of the UAV is identified and reconstructed based on the feature association weight matrix, and then the UAV is driven to perform intelligent fault identification based on the identified and reconstructed inspection path.

[0011] In this embodiment, a non-dominated sorting genetic algorithm is used to construct a dual-objective path model based on historical fault data of the target inspection area.

[0012] In this embodiment, the multi-source perception data includes visible light image data, infrared thermal imaging data and three-dimensional spatial structure data, and the multi-source perception data of the target inspection area is collected using the drone's visible light camera, infrared thermal imager and lidar.

[0013] In this embodiment, the dual-objective path model dynamically optimizes the inspection path of the drone based on the multi-source perception data, and the optimized path and perception area obtained during the drone inspection specifically include:

[0014] Determining an initial inspection path of the UAV using the multi-source perception data;

[0015] Perform risk identification and inspection efficiency optimization on the multi-source perception data to obtain dual optimization goals;

[0016] The non-dominated sorting genetic algorithm is used to iteratively optimize the initial inspection path based on the dual optimization objectives to obtain the optimized path and perception area during the drone inspection.

[0017] In this embodiment, the multi-source perception data is hierarchically identified by the cloud collaborative platform of the drone to obtain the state prediction value of the drone, specifically including:

[0018] The multi-source perception data are aligned and fused according to timestamps through the UAV's cloud collaboration platform to obtain a multimodal input feature tensor;

[0019] Performing fine-grained recognition on the multimodal input feature tensor to obtain deep semantic features;

[0020] The state change trend of the UAV is determined based on the spatiotemporal position of the target inspection area and the deep semantic features, and the state prediction value of the UAV is determined based on the state change trend.

[0021] In this embodiment, constructing an intelligent path guidance function for drone inspection based on the optimized path, the perception area, and the state prediction value specifically includes:

[0022] The spatial coordinate sequence in the optimization path, the risk score matrix of the perception area and the state prediction value are jointly encoded into a path guidance input vector;

[0023] Utilize a deep learning model based on the attention mechanism to construct a path guidance function;

[0024] determining a path guidance vector using the optimized path and the sensing area;

[0025] The state prediction value is converted into a state prediction vector, and then the path guidance function is supervised and trained using the state prediction vector and the path guidance vector to obtain an intelligent path guidance function for drone inspection.

[0026] In this embodiment, the intelligent path guidance function is back-propagated and corrected to obtain the fault type characteristics and detection accuracy during drone inspection, specifically including:

[0027] Obtaining the fault location and fault type during drone inspection, predicting and identifying the fault location and fault type through the intelligent path guidance function, and obtaining a path offset and a fault detection deviation;

[0028] determining a penalty factor based on the path offset and the fault detection bias;

[0029] Performing gradient backpropagation on the intelligent path guidance function through the penalty factor to obtain the intelligent path guidance function after weight parameter adjustment;

[0030] The intelligent path guidance function after weight parameter adjustment is used to correct and identify the fault location and the fault type, and obtain the fault type characteristics and detection accuracy during drone inspection.

[0031] In this embodiment, the cloud collaboration platform of the drone is an intelligent computing platform based on deep learning and incremental training.

[0032] In this embodiment, the dual-objective path model is a path optimization model constructed by a dual-objective function, and the dual-objective function includes inspection time and abnormal area coverage.

[0033] In a second aspect, the present application provides a drone inspection fault intelligent identification system for executing a drone inspection fault intelligent identification method, the intelligent identification system comprising:

[0034] A dual-target path model building module is used to build a dual-target path model based on historical fault data in the target inspection area;

[0035] A path optimization module is used to use the drone to collect multi-source perception data of the target inspection area. The dual-target path model dynamically optimizes the drone's inspection path based on the multi-source perception data to obtain the optimized path and perception area during the drone inspection;

[0036] A path guidance module is used to perform hierarchical recognition of the multi-source perception data through the UAV's cloud collaborative platform to obtain a state prediction value of the UAV, and to construct an intelligent path guidance function for the UAV inspection based on the optimized path, the perception area, and the state prediction value;

[0037] A weight construction module is used to perform back-propagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during the drone inspection, and to construct a feature association weight matrix of the inspection path based on the fault type characteristics and the detection accuracy;

[0038] The path reconstruction execution module is used to identify and reconstruct the inspection path of the drone based on the feature association weight matrix, and then drive the drone to perform intelligent fault identification based on the identified and reconstructed inspection path.

[0039] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0040] A dual-target path model is constructed based on historical fault data of the target inspection area; multi-source perception data of the target inspection area is collected by a drone, and the dual-target path model dynamically optimizes the drone's inspection path based on the multi-source perception data to obtain the optimized path and perception area during the drone inspection; the multi-source perception data is hierarchically identified through the drone's cloud-based collaborative platform to obtain the drone's state prediction value, and an intelligent path guidance function for the drone inspection is constructed based on the optimized path, the perception area and the state prediction value; the intelligent path guidance function is back-propagated and corrected to obtain the fault type characteristics and detection accuracy during the drone inspection, and a feature association weight matrix of the inspection path is constructed based on the fault type characteristics and the detection accuracy; the drone's inspection path is identified and reconstructed based on the feature association weight matrix, thereby driving the drone to perform intelligent fault identification based on the identified and reconstructed inspection path.

[0041] It can be seen that in this application, path reconstruction and intelligent fault identification driven by recognition feedback can be realized. First, by adopting a non-dominated sorting genetic algorithm and constructing a dual-target path model based on the historical fault data of the target inspection area, the coverage efficiency of the inspection path and the attention weight of the high-risk area can be taken into account at the same time, thereby realizing multi-target intelligent modeling of the initial inspection path and providing a structural priori reference for the subsequent dynamic optimization of the path; secondly, the multi-source perception data of the target area is collected by the drone, including visible light images, infrared thermal imaging and spatial point cloud information, and combined with the dual-target path model to realize dynamic optimization processing of the path, which can not only adapt to environmental changes in real time, but also ensure multi-dimensional coverage of key areas, thereby improving the integrity of the inspection data and the dynamic response capability of the path; then, the multi-source perception data is layered and identified through the cloud-based collaborative platform of the drone, and combined with the optimized path , perception area and state prediction value to construct an intelligent path guidance function, which can make the path planning process have state perception ability, realize adaptive path scheduling under recognition guidance, and improve the correlation between recognition area and path node; then, by obtaining the fault detection deviation and path offset, perform back propagation correction of the intelligent path guidance function, and combine the detection accuracy and fault type to construct the feature association weight matrix of the inspection path, which can not only provide feedback and optimize the path guidance strategy, but also quantify the mapping relationship between recognition risk and path priority, and construct a recognition-driven path control model; finally, the inspection path is identified and reconstructed based on the feature association weight matrix, and the UAV is driven to perform fault detection tasks according to the identified and reconstructed path, thereby realizing efficient coupling of path planning and recognition decision-making, and significantly improving the fault detection accuracy and path adaptability of the UAV in a dynamic environment.

[0042] In summary, the technical solution adopted in this application can realize path reconstruction and intelligent fault identification based on recognition feedback drive to improve the accuracy of drone inspection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 This is a flow chart of a method for intelligently identifying faults during drone inspections provided by this application;

[0045] Figure 2 This is an exemplary flow chart for determining the optimized path and sensing area for drone inspections according to the present application;

[0046] Figure 3 is an exemplary flow chart for determining a state prediction value of a drone according to the present application;

[0047] Figure 4 This is a module structure diagram of the intelligent recognition system provided by this application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0049] An embodiment of the present application provides a system and method for intelligent fault identification during drone inspection, the core of which is to construct a dual-target path model based on historical fault data of a target inspection area; use a drone to collect multi-source perception data of a target inspection area, and the dual-target path model dynamically optimizes the drone's inspection path based on the multi-source perception data to obtain an optimized path and perception area during the drone inspection; perform hierarchical identification on the multi-source perception data through the drone's cloud-based collaborative platform to obtain a state prediction value of the drone, and construct an intelligent path guidance function for the drone inspection based on the optimized path, the perception area, and the state prediction value; perform backpropagation correction on the intelligent path guidance function to obtain fault type characteristics and detection accuracy during the drone inspection, and construct a feature association weight matrix of the inspection path based on the fault type characteristics and the detection accuracy; identify and reconstruct the drone's inspection path based on the feature association weight matrix, and then drive the drone to perform intelligent fault identification based on the identified and reconstructed inspection path.

[0050] Example 1: In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods. Figure 1 As shown in FIG, this figure is an exemplary flow chart of a method for intelligently identifying faults during drone inspection according to this embodiment of the present application. The intelligent identification method includes the following steps:

[0051] In step S1, a dual-target path model is constructed based on historical fault data of the target inspection area.

[0052] In the specific implementation, a non-dominated sorting genetic algorithm is used to construct a dual-objective path model based on the historical fault data of the target inspection area, namely: first, the historical fault data of the target inspection area can be obtained through the database of the target inspection area, and the historical fault data includes: geographical location information, time label, fault type and fault degree; then, a fault density map is constructed based on the historical fault data through a visualization algorithm, and the fault density map is used to reflect the spatial distribution characteristics of the target inspection area; and the target inspection area is discretized into several inspection nodes through a random forest algorithm, and a weighted directed graph is constructed with the geographical distance between the inspection nodes and the historical fault density of the area to which the nodes belong; then, minimizing the total inspection path time is used as the first objective function, and the coverage rate of the historical high fault density area is used as the second objective function to obtain a dual-objective function, and the weighted directed graph is optimized by the non-dominated sorting genetic algorithm and the dual-objective function, and the optimized graph model is used as the dual-objective path model.

[0053] It should be noted that the dual-objective path model in this embodiment is a path optimization model constructed by a dual-objective function, which includes inspection time and abnormal area coverage, which is conducive to improving the execution efficiency of inspection tasks and the coverage capability of key hidden danger areas; wherein, by constructing a weighted directed graph to represent the relationship between inspection nodes, and based on the historical fault density as a quantitative indicator of abnormal area coverage, the model can adapt to the distribution characteristics of key areas of the inspection task; in addition, the non-dominated sorting genetic algorithm has a strong global optimization capability and the ability to adapt to multi-objective optimization problems, and can achieve effective trade-offs between multiple inspection requirements, thereby outputting an optimal inspection path set with practical guiding significance.

[0054] In step S2, a drone is used to collect multi-source perception data of the target inspection area. The dual-target path model dynamically optimizes the drone's inspection path based on the multi-source perception data to obtain the optimized path and perception area during the drone inspection.

[0055] It should be noted that in this application, the multi-source perception data includes visible light image data, infrared thermal imaging data and three-dimensional spatial structure data. The visible light camera, infrared thermal imager and lidar of the drone are used to collect multi-source perception data of the target inspection area, wherein the visible light image data is used to identify visual features such as appearance damage and obstructions, the infrared thermal imaging data is used to identify thermal features such as temperature anomalies and hot spot concentration, and the three-dimensional spatial structure data is used to identify geometric features such as structural deformation, distance measurement and spatial layout; through multi-source perception data, high-dimensional features of the target inspection area can be constructed, and the system's comprehensive perception ability of abnormal risks in complex environments can be enhanced, providing high-quality data support for subsequent path optimization, risk modeling and decision analysis; in addition, the visible light camera, infrared thermal imager and lidar of the drone are used to collect multi-source perception data of the target inspection area, the visible light camera is used to collect visible light image data, the infrared thermal imager is used to collect infrared thermal imaging data, and the lidar is used to collect three-dimensional spatial structure data, which is conducive to the needs of carrying out multi-scene inspection tasks in different types of target areas.

[0056] In specific implementation, visible light image data, infrared thermal imaging data and laser point cloud data of the target inspection area are obtained by using a drone equipped with a visible light camera, an infrared thermal imager and a lidar; wherein, the visible light camera can collect data with a resolution of 1080p, which is used to provide surface structure and texture information of the target area; the infrared thermal imager collects thermal image data of the target area by sensing thermal radiation, and can identify temperature anomalies or heating equipment; the lidar collects three-dimensional spatial structure data based on the time-of-flight principle, obtains the three-dimensional coordinates and reflection intensity information of the surface points of the target area, and finally maps the collected visible light image data, infrared thermal imaging data and three-dimensional spatial structure data into multi-source fusion perception data through the existing projection matrix.

[0057] Preferably, in this embodiment, reference Figure 2 As shown in the figure, this is an exemplary flow chart for determining the optimized path and perception area during drone inspection according to the present application. In this embodiment, the dual-objective path model dynamically optimizes the drone inspection path based on the multi-source perception data, and the optimized path and perception area during drone inspection can be obtained by the following steps:

[0058] First, in step S21, the initial inspection path of the UAV is determined by the multi-source sensing data;

[0059] Then, in step S22, risk identification and inspection efficiency optimization are performed on the multi-source perception data to obtain dual optimization objectives;

[0060] Finally, in step S23, the initial inspection path is iteratively optimized based on the dual optimization objectives using a non-dominated sorting genetic algorithm to obtain the optimized path and perception area during the drone inspection.

[0061] In the specific implementation, first, the multi-source perception data is spatially rasterized and the target inspection area is divided into multiple perception units, where the perception unit corresponds to a set of perception data containing visible light images, infrared thermal imaging images and three-dimensional structural information. The regional coverage algorithm is used to perform spatial distribution identification on all perception units to obtain the initial inspection path of the drone; then, the different modal input data are processed for time synchronization and spatial alignment to form a unified input tensor, and the pre-trained convolutional neural network (CNN) and long short-term memory network (LSTM) are used to extract the space-time feature vector, and it is input into the classification discrimination layer to output the abnormal category probability of each perception unit; at the same time, according to indicators such as the flight length, inspection time and area coverage in the inspection flight path, an inspection efficiency function is constructed to quantify the path execution efficiency, thereby converting the inspection efficiency function into a unified input tensor. and abnormal category probability as dual optimization objectives; finally, the abnormal category probability and the inspection efficiency function are used as constraint parameters of the non-dominated sorting genetic algorithm, so that the loss function of the non-dominated sorting genetic algorithm with the constraint parameters set is used as a dual-objective optimization function. In the optimization process, the encoding method uses the perception unit access order as the gene structure, and generates multiple path candidate solutions through crossover and mutation operations. The risk identification value and inspection efficiency value of each solution are calculated, and non-dominated sorting and population update are performed according to the Pareto optimal principle to obtain the optimized path and perception area during drone inspection; it should be noted that the crossover operation can realize path recombination by exchanging two parent path gene fragments, and the mutation operation breaks the local optimal trap by adjusting the access order, thereby improving the solution space exploration ability, and the optimization result is controlled by the elite retention strategy to retain high-quality solutions and ensure evolutionary stability.

[0062] It should be noted that the dual-objective path model avoids the problem of missing high-risk areas that may be caused by simply taking the shortest path or the shortest time as the goal by integrating the two optimization dimensions of risk identification and path efficiency. Among them, the dual-objective path optimization implemented by the non-dominated sorting genetic algorithm has a stronger diversity retention ability and higher global convergence performance, and can adaptively adjust the flight path in a complex inspection environment; in addition, the optimized path contains flight sequence information, which facilitates the adaptive allocation of tasks.

[0063] In step S3, the multi-source perception data is hierarchically identified through the cloud-based collaborative platform of the drone to obtain the state prediction value of the drone, and an intelligent path guidance function for drone inspection is constructed based on the optimized path, the perception area and the state prediction value.

[0064] It should be noted that in this application, the cloud-based collaborative platform for drones is an intelligent computing platform based on deep learning and incremental training, which has a knowledge distillation-based distribution mechanism and recognition result fusion capability. The cloud-based collaborative platform for drones is a heterogeneous fusion system integrating data processing, model training and task scheduling. The cloud-based collaborative platform has the ability to efficiently process multi-source perception data uploaded by drones during flight, and can realize remote reasoning deployment and online iterative optimization of models; wherein, the cloud-based collaborative platform performs incremental training on various perception models, that is, based on the original training model, small batches of rapid learning and updating are performed according to newly collected data, which can avoid full retraining and improve model adaptability and update efficiency; in addition, the cloud-based collaborative platform includes a spatiotemporal synchronous modeling algorithm, a multimodal deep fusion recognition network and a spatiotemporal evolution modeling network. Based on the cloud-based collaborative platform, resource scheduling and model fusion can be performed on the task status of multiple drones to ensure data consistency under multi-machine collaborative operations.

[0065] Preferably, in this embodiment, reference Figure 3 As shown in FIG, this figure is an exemplary flow chart for determining the state prediction value of a drone according to the present application. In this embodiment, the multi-source perception data is hierarchically identified by the drone's cloud collaborative platform to obtain the state prediction value of the drone. Specifically, the following steps can be used to achieve this:

[0066] First, in step S31, the multi-source perception data are aligned and fused according to timestamps through the cloud collaboration platform of the drone to obtain a multimodal input feature tensor;

[0067] Then, in step S32, fine-grained recognition is performed on the multimodal input feature tensor to obtain deep semantic features;

[0068] Finally, in step S33, the state change trend of the drone is determined based on the spatiotemporal position of the target inspection area for the deep semantic features, and the state prediction value of the drone is further determined based on the state change trend.

[0069] In the specific implementation, first, the cloud-based collaborative platform aligns and fuses the multi-source perception data according to timestamps through a spatiotemporal synchronous modeling algorithm to obtain a multimodal input feature tensor, which includes: spatiotemporal sequence information, spatial structure information, and thermal anomaly response information; then, the multimodal input feature tensor is fine-grainedly recognized through the multimodal deep fusion recognition network of the cloud-based collaborative platform, and the output features of the fine-grained recognition are used as deep semantic features, wherein the multimodal deep fusion recognition network includes: convolutional neural networks, point cloud neural networks, and multi-head self-attention mechanisms, which can extract features including structural defects. Finally, based on the spatiotemporal evolution modeling network of the cloud collaborative platform, the deep semantic features are combined with the spatiotemporal position of the target inspection area to model the state change trend of the UAV under specific spatiotemporal conditions, and then the task delay parameters in the state change trend are analyzed through the spatiotemporal evolution modeling network, and then the task delay parameters are used as the state prediction value of the UAV. Among them, the spatiotemporal evolution modeling network can be constructed based on the Transformer architecture, which is conducive to maintaining spatial consistency and temporal continuity, thereby outputting the state change trend of the UAV.

[0070] It should be noted that the state prediction value in this application is based on the dynamic evolution characteristics during the task execution process and the spatiotemporal modeling based on deep semantic features, and outputs task delay data related to a certain time period in the future. It can be used to quantitatively evaluate the operating capabilities and potential risks of drones in subsequent inspection tasks. That is, according to the predicted task delay, the task priority or perception strategy can be dynamically adjusted to achieve efficient resource allocation.

[0071] In this embodiment, the intelligent path guidance function for drone inspection based on the optimized path, the perception area, and the state prediction value can be constructed in the following manner, namely:

[0072] The spatial coordinate sequence in the optimization path, the risk score matrix of the perception area and the state prediction value are jointly encoded into a path guidance input vector;

[0073] Utilize a deep learning model based on the attention mechanism to construct a path guidance function;

[0074] determining a path guidance vector using the optimized path and the sensing area;

[0075] The state prediction value is converted into a state prediction vector, and then the path guidance function is supervised and trained using the state prediction vector and the path guidance vector to obtain an intelligent path guidance function for drone inspection.

[0076] In the specific implementation, first, the spatial coordinate sequence in the optimized path is extracted through the dual-objective path model, and the dual-objective path model is used to construct a risk score matrix for the perception area. The state prediction value, the spatial coordinate sequence and the risk score matrix are standardized and converted into a path guidance input vector through one-hot encoding; secondly, the attention mechanism is preset based on the computational complexity of the standardization process, and the path guidance function is constructed based on the Transformer network; then, the Transformer network is trained twice by optimizing the path and the perception area, and the path guidance vector is obtained by the Transformer network after secondary training through the real-time inspection path prediction of the UAV; finally, the state prediction vector is input and mapped to the feature dimension consistent with the path guidance vector, and the path guidance function is supervised and trained using cross entropy as the loss function. The network weights are optimized through the back propagation algorithm during the supervised training process, and the intelligent path guidance function during the UAV inspection can be obtained.

[0077] It should be noted that the intelligent path guidance function constructed in this embodiment has the ability of adaptive dynamic adjustment. It can respond to path changes in real time during operation, which facilitates the realization of optimal decision guidance; the intelligent path guidance function has the ability to fuse high-dimensional heterogeneous inputs, which can effectively extract deep semantic associations between data from different sources, and significantly improve the rationality of drone inspection path selection; in addition, by introducing the attention mechanism, the intelligent path guidance function can focus on key areas and high-risk targets in the current inspection task, thereby improving the overall path guidance accuracy; the intelligent path guidance function can be deployed on the drone's cloud-based collaborative platform, and real-time path optimization can be performed through the drone's wireless communication function.

[0078] In step S4, the intelligent path guidance function is back-propagated and corrected to obtain the fault type characteristics and detection accuracy during the drone inspection, and a feature association weight matrix of the inspection path is constructed based on the fault type characteristics and the detection accuracy;

[0079] In this embodiment, the intelligent path guidance function is back-propagated and corrected to obtain the fault type characteristics and detection accuracy during the drone inspection, specifically in the following manner, namely:

[0080] Obtaining the fault location and fault type during drone inspection, predicting and identifying the fault location and fault type through the intelligent path guidance function, and obtaining a path offset and a fault detection deviation;

[0081] determining a penalty factor based on the path offset and the fault detection bias;

[0082] Performing gradient backpropagation on the intelligent path guidance function through the penalty factor to obtain the intelligent path guidance function after weight parameter adjustment;

[0083] The intelligent path guidance function after weight parameter adjustment is used to correct and identify the fault location and the fault type, and obtain the fault type characteristics and detection accuracy during drone inspection.

[0084] In the specific implementation, first, the multimodal data transmitted by the wireless communication function book of the drone is divided into the fault location and fault type by the recognition function pre-trained based on manual annotation, and the intelligent path guidance function is used to predict and output the fault location and fault type, and then the spatial offset distance between the predicted location and the actual fault location is used as the path offset, and the category difference between the predicted fault label and the actual label is used as the fault detection deviation, wherein the category difference can be measured by classification cross entropy; secondly, the weighted sum of the path offset and the fault detection deviation is used as a penalty factor. Preferably, the weight of the weighted sum can be based on the validation set using the Bayesian optimization method, traversing different weight combinations, and evaluating the inspection accuracy of the model, the average path offset value and other indicators, and finally selecting the optimal weight. In other embodiments, the weight can also be determined by other methods, which are not limited here; then, The penalty factor is substituted into the path guidance function as a constant term to obtain a new expression, and the parameters in the new expression are gradient reverse updated through the back-propagation algorithm. The intelligent path guidance function after weight parameter adjustment can be obtained through the above update process. Preferably, the gradient reverse update process can use a gradient descent optimizer to adjust the multi-source input feature channel weights and the attention mechanism weights, which is beneficial to enhancing the model's perception of the spatial distribution and type differences of faults. In other embodiments, other optimizers with the same adjustment effect can also be used for adjustment, which is not limited here; finally, the intelligent path guidance function after weight adjustment is used to predict and identify the fault location and fault type obtained by the drone, thereby outputting the fault type characteristics and detection accuracy during the drone inspection, extracting the fault type characteristics and calculating the detection accuracy, wherein the detection accuracy can be evaluated using the recall rate.

[0085] It should be noted that the intelligent path guidance function in this embodiment can plan and dynamically schedule the UAV path; by establishing a penalty factor mechanism through path offset and fault detection deviation, it can effectively identify the source of error in the model in spatial positioning and type judgment, thereby realizing dynamic correction during the training process, enhancing the adaptability of fault characteristics, and facilitating driving the UAV to perform intelligent fault inspection; in addition, the intelligent path guidance function is updated in real time through the backpropagation correction process, and the parameters can be adjusted in combination with the real-time perception results, so that the guidance function can continuously adapt to environmental changes and diversified fault forms, thereby ensuring the recognition accuracy of the UAV during long-term, multi-task inspections.

[0086] In this embodiment, the feature association weight matrix of the inspection path is constructed based on the fault type characteristics and the detection accuracy in the following manner, namely:

[0087] Determine a risk level score based on the fault type characteristics;

[0088] Determining detection confidence based on the detection accuracy and the risk level score;

[0089] A weight matrix is ​​constructed based on the node features of the inspection path, and then the weight matrix is ​​associated and optimized according to the detection confidence to obtain a feature association weight matrix of the inspection path.

[0090] In specific implementation, first, the fault type characteristics include fault category, fault location and fault degree, and multiple dimensions of the fault type characteristics are encoded into feature vectors, and then the risk level is regressed and analyzed based on historical experience and the feature vectors to obtain a risk level score; then, the detection accuracy and the inverse of the risk level score are multiplied, and the product obtained is used as the detection confidence, wherein the detection confidence represents the reliability of the fault identification result. The higher the detection confidence value, the stronger the reliability of the fault identification result, which can be given priority in subsequent decisions such as path scheduling, task planning, and risk warning. The lower the detection confidence, the more response measures such as downgrade processing, backup paths, and enhanced inspections can be taken; finally, for unmanned The node with faults identified in the inspection path of the machine is feature-encoded to obtain node features, wherein the node features include node position and fault type, and the node position and fault type can be converted into node feature vectors through one-hot encoding, and then the matrix composed of the node feature vectors arranged in rows is used as the weight matrix, and then each row of the weight matrix is ​​weight-associated respectively through the detection confidence, wherein the detection confidence only assigns weights to the fault types, that is, in the weight matrix, the fault types in the weight matrix corresponding to each row are all given weights by the detection confidence, and the matrix obtained after weight assignment is used as the feature association weight matrix of the inspection path, which can be used for dynamic planning in the path reconstruction stage, and can also be used as the adjacency weight matrix of the graph neural network input.

[0091] It should be noted that the feature association weight matrix in this application refers to the multi-dimensional features such as spatial information, task attributes, fault type characteristics and detection confidence of each path node in the UAV inspection path, and is constructed by feature similarity calculation and confidence weighted optimization. It is an expression matrix of the correlation strength between path nodes, which can be used to quantify the risk transmission ability between different nodes in the inspection task; in addition, by optimizing the weight distribution between nodes through detection confidence, high-risk nodes with higher identification accuracy can be given priority in the path planning process, thereby improving the reliability and efficiency of the overall inspection strategy.

[0092] In step S5, the inspection path of the UAV is identified and reconstructed based on the feature association weight matrix, and then the UAV is driven to perform intelligent fault identification based on the identified and reconstructed inspection path.

[0093] In this embodiment, the inspection path of the UAV is identified and reconstructed based on the feature association weight matrix, and then the UAV is driven to perform intelligent fault identification based on the identified and reconstructed inspection path. Specifically, the following method can be used, namely:

[0094] The feature association weight matrix is ​​mapped to the node features of the inspection path, and then the reconstructed path sequence of the inspection path is determined based on the particle swarm optimization algorithm;

[0095] The cloud-based collaborative platform of the UAV drives the UAV to perform intelligent fault identification based on the reconstructed inspection path according to the reconstructed path sequence.

[0096] In the specific implementation, the feature association weight matrix is ​​first mapped to the node feature space of the inspection path, and the association fusion of the weight matrix and the node features is realized by constructing a weight mapping function to obtain a weighted feature representation. Based on the weighted feature representation, the particle swarm optimization algorithm is used to perform global search and local adjustment on the inspection path, and then the optimal access sequence of the obtained path nodes is used as the reconstructed path sequence of the inspection path; wherein, the fitness function of the particle swarm optimization algorithm comprehensively considers the path length, node risk weight and detection confidence to ensure that the path not only meets the inspection efficiency but also gives priority to covering high-risk areas; in actual implementation, multiple particles can be initialized to represent different path sequences, and the speed and position of the particles are iteratively updated to gradually approach the optimal path solution until the convergence conditions are met or the preset number of iterations is reached; then, the cloud-based collaborative platform of the drone reconstructs the path sequence and dynamically adjusts the fault identification path to the drone end based on deep learning and incremental training mechanisms, and then drives the drone to perform intelligent fault identification based on the dynamically adjusted fault identification path through wireless drive technology.

[0097] It should be noted that the inspection path identification and reconstruction in this application refers to the reordering and structural adjustment of the original path node sequence through the optimization algorithm based on the risk correlation and detection confidence information between nodes provided by the feature association weight matrix, so as to generate an inspection path sequence that meets the current perception capability, which is conducive to improving the path decision-making ability of the drone in complex inspection scenarios, and making the path planning results structurally reasonable. Among them, the reconstructed path sequence can be synchronously transmitted to the cloud-based collaborative platform, and combined with the incremental learning mechanism to optimize the recognition model, so that the drone-side fault identification strategy can be updated in real time and dynamically adapt to the changing mission objectives, thereby improving the overall intelligence level of the system.

[0098] It can be seen that in this application, path reconstruction and intelligent fault identification driven by recognition feedback can be realized. First, by adopting a non-dominated sorting genetic algorithm and constructing a dual-target path model based on the historical fault data of the target inspection area, the coverage efficiency of the inspection path and the attention weight of the high-risk area can be taken into account at the same time, thereby realizing multi-target intelligent modeling of the initial inspection path and providing a structural priori reference for the subsequent dynamic optimization of the path; secondly, the multi-source perception data of the target area is collected by the drone, including visible light images, infrared thermal imaging and spatial point cloud information, and combined with the dual-target path model to realize dynamic optimization processing of the path, which can not only adapt to environmental changes in real time, but also ensure multi-dimensional coverage of key areas, thereby improving the integrity of the inspection data and the dynamic response capability of the path; then, the multi-source perception data is layered and identified through the cloud-based collaborative platform of the drone, and combined with the optimized path , perception area and state prediction value to construct an intelligent path guidance function, which can make the path planning process have state perception ability, realize adaptive path scheduling under recognition guidance, and improve the correlation between recognition area and path node; then, by obtaining the fault detection deviation and path offset, perform back propagation correction of the intelligent path guidance function, and combine the detection accuracy and fault type to construct the feature association weight matrix of the inspection path, which can not only provide feedback and optimize the path guidance strategy, but also quantify the mapping relationship between recognition risk and path priority, and construct a recognition-driven path control model; finally, the inspection path is identified and reconstructed based on the feature association weight matrix, and the UAV is driven to perform fault detection tasks according to the identified and reconstructed path, thereby realizing efficient coupling of path planning and recognition decision-making, and significantly improving the fault detection accuracy and path adaptability of the UAV in a dynamic environment.

[0099] In summary, the technical solution adopted in this application can realize path reconstruction and intelligent fault identification based on recognition feedback drive to improve the accuracy of drone inspection results.

[0100] In the second embodiment, the present application provides a UAV inspection fault intelligent identification system, referring to Figure 4As shown in FIG, this figure is a module structure diagram of the intelligent recognition system shown in this embodiment of the present application, and the intelligent recognition system includes:

[0101] A dual-target path model construction module 100 is used to construct a dual-target path model based on historical fault data of a target inspection area;

[0102] The path optimization module 200 is used to collect multi-source perception data of the target inspection area using the drone. The dual-target path model dynamically optimizes the drone's inspection path based on the multi-source perception data to obtain the optimized path and perception area during the drone inspection.

[0103] The path guidance module 300 is used to perform hierarchical recognition of the multi-source perception data through the UAV's cloud collaborative platform to obtain a state prediction value of the UAV, and to construct an intelligent path guidance function for the UAV inspection based on the optimized path, the perception area, and the state prediction value;

[0104] A weight construction module 400 is used to perform back-propagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during the drone inspection, and to construct a feature association weight matrix of the inspection path based on the fault type characteristics and the detection accuracy;

[0105] The path reconstruction execution module 500 is used to identify and reconstruct the inspection path of the drone based on the feature association weight matrix, and then drive the drone to perform intelligent fault identification based on the identified and reconstructed inspection path.

[0106] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0107] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, magnetic disk storage, or magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0108] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

Claims

1. A method for intelligently identifying faults during drone inspection, characterized in that: The intelligent identification method comprises the following steps: A dual-target path model is constructed based on historical fault data in the target inspection area; The dual-target path model dynamically optimizes the inspection path of the drone based on the multi-source perception data to obtain the optimized path and perception area of ​​the drone during inspection. The multi-source perception data is hierarchically identified through the UAV's cloud collaborative platform to obtain a state prediction value of the UAV, and an intelligent path guidance function for the UAV inspection is constructed based on the optimized path, the perception area, and the state prediction value; Performing back-propagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during the drone inspection, and constructing a feature association weight matrix of the inspection path based on the fault type characteristics and the detection accuracy; Identify and reconstruct the inspection path of the UAV based on the feature association weight matrix, and then drive the UAV to perform intelligent fault identification based on the identified and reconstructed inspection path; The state prediction value is a task delay parameter, which is used to evaluate the operation capability of the UAV in the inspection task; The intelligent path guidance function for constructing the drone inspection based on the optimized path, the perception area, and the state prediction value specifically includes: The spatial coordinate sequence in the optimization path, the risk score matrix of the perception area, and the state prediction value are jointly encoded into a path guidance input vector, and then a deep learning model based on the attention mechanism is used to construct a path guidance function. determining a path guidance vector using the optimized path and the sensing area; Converting the state prediction value into a state prediction vector, and then performing supervised training on the path guidance function using the state prediction vector and the path guidance vector to obtain an intelligent path guidance function for drone inspection; Among them, the intelligent path guidance function is back-propagated and corrected to obtain the fault type characteristics and detection accuracy during drone inspection, specifically including: Obtaining the fault location and fault type during drone inspection, predicting and identifying the fault location and fault type through the intelligent path guidance function, and obtaining a path offset and a fault detection deviation; determining a penalty factor based on the path offset and the fault detection bias; Performing gradient backpropagation on the intelligent path guidance function through the penalty factor to obtain the intelligent path guidance function after weight parameter adjustment; The intelligent path guidance function after weight parameter adjustment is used to correct and identify the fault location and the fault type, and obtain the fault type characteristics and detection accuracy during the drone inspection; The feature association weight matrix of the inspection path constructed based on the fault type characteristics and the detection accuracy specifically includes: Determine a risk level score based on the fault type characteristics; Determining detection confidence based on the detection accuracy and the risk level score; A weight matrix is ​​constructed based on the node features of the inspection path, and then the weight matrix is ​​associated and optimized according to the detection confidence to obtain a feature association weight matrix of the inspection path.

2. The intelligent fault identification method for drone inspection according to claim 1, characterized in that: A non-dominated sorting genetic algorithm is used to construct a dual-objective path model based on the historical fault data of the target inspection area.

3. The intelligent identification method for UAV inspection faults according to claim 1, characterized in that: The multi-source perception data includes visible light image data, infrared thermal imaging data and three-dimensional spatial structure data, and the multi-source perception data of the target inspection area is collected using the drone's visible light camera, infrared thermal imager and lidar.

4. The intelligent identification method for UAV inspection faults according to claim 1, characterized in that: The dual-objective path model dynamically optimizes the inspection path of the UAV based on the multi-source perception data, and obtains the optimized path and perception area during the UAV inspection, specifically including: Determining an initial inspection path of the UAV using the multi-source perception data; Perform risk identification and inspection efficiency optimization on the multi-source perception data to obtain dual optimization goals; The non-dominated sorting genetic algorithm is used to iteratively optimize the initial inspection path based on the dual optimization objectives to obtain the optimized path and perception area during the drone inspection.

5. The intelligent identification method for UAV inspection faults according to claim 1, characterized in that: The multi-source perception data is hierarchically identified through the UAV's cloud collaborative platform to obtain the UAV's state prediction value, which specifically includes: The multi-source perception data are aligned and fused according to timestamps through the UAV's cloud collaboration platform to obtain a multimodal input feature tensor; Performing fine-grained recognition on the multimodal input feature tensor to obtain deep semantic features; The state change trend of the UAV is determined based on the spatiotemporal position of the target inspection area and the deep semantic features, and the state prediction value of the UAV is determined based on the state change trend.

6. The intelligent identification method for UAV inspection faults according to claim 1, characterized in that: The cloud-based collaborative platform for drones is an intelligent computing platform based on deep learning and incremental training.

7. The intelligent identification method for UAV inspection faults according to claim 1, characterized in that: The dual-objective path model is a path optimization model constructed by dual-objective functions, and the dual-objective functions include inspection time and abnormal area coverage.

8. An intelligent identification system for UAV inspection faults, used to execute the intelligent identification method for UAV inspection faults according to any one of claims 1 to 7, characterized in that: The intelligent recognition system includes: A dual-target path model building module is used to build a dual-target path model based on historical fault data in the target inspection area; A path optimization module is used to use the drone to collect multi-source perception data of the target inspection area. The dual-target path model dynamically optimizes the drone's inspection path based on the multi-source perception data to obtain the optimized path and perception area during the drone inspection; A path guidance module is used to perform hierarchical recognition of the multi-source perception data through the UAV's cloud collaborative platform to obtain a state prediction value of the UAV, and to construct an intelligent path guidance function for the UAV inspection based on the optimized path, the perception area, and the state prediction value; A weight construction module is used to perform back-propagation correction on the intelligent path guidance function to obtain the fault type characteristics and detection accuracy during the drone inspection, and to construct a feature association weight matrix of the inspection path based on the fault type characteristics and the detection accuracy; The path reconstruction execution module is used to identify and reconstruct the inspection path of the drone based on the feature association weight matrix, and then drive the drone to perform intelligent fault identification based on the identified and reconstructed inspection path.

Images