Dynamic inspection method and device based on unmanned aerial vehicle, electronic equipment and storage medium

Abstract

The invention discloses a dynamic inspection method and device based on an unmanned aerial vehicle, electronic equipment and a storage medium, and relates to the technical field of software and platforms or other related technical fields, and the method comprises the steps: carrying out the three-dimensional scene modeling of a to-be-inspected region through the space-time alignment data obtained through preprocessing; acquiring an inspection operation instruction, and performing inspection task distribution of multiple flight constraints on the dynamic environment model obtained by modeling to obtain an optimized task execution scheme; and performing global path planning and local space-time cooperative path optimization on the task execution scheme based on the high-precision three-dimensional scene model obtained by modeling to obtain a target inspection path, thereby remotely controlling the target unmanned aerial vehicle to perform optimized path inspection and anomaly analysis according to the target inspection path, and generating an anomaly inspection report of the target inspection area. According to the invention, the technical problems of poor adaptability and low accuracy of the inspection result of the inspection mode of planning the inspection path based on historical data in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of software and platform technology or other related technical fields. Specifically, it relates to a dynamic inspection method and device, electronic equipment and storage medium based on unmanned aerial vehicles (UAVs). Background Technology

[0002] With the rapid development of drone technology, its application in fields such as aerial surveying and mapping, power line inspection, and agricultural plant protection is becoming increasingly widespread. Traditional drone control methods mainly rely on manual remote control or autonomous flight along preset routes, which suffers from problems such as complex operation, low flexibility, and poor environmental adaptability.

[0003] In recent years, with the rapid development of artificial intelligence and big data technologies, some technologies use historical inspection data for path calculation and planning. This allows computers to plan the flight path of drones in advance and remotely control the drones to fly along the preset path and inspect the target area. However, this inspection method has poor adaptability. As time goes by, the surrounding environment may change, resulting in low accuracy of path planning results. Consequently, the usability of a single inspection result is low, making it difficult to achieve the purpose of effective inspection.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a dynamic inspection method and device, electronic device and storage medium based on unmanned aerial vehicles (UAVs), to at least solve the technical problems in related technologies where the inspection method based on historical data to plan inspection paths has poor adaptability and low accuracy of inspection results.

[0006] According to one aspect of the present invention, a dynamic inspection method based on an unmanned aerial vehicle (UAV) is provided, comprising: acquiring multi-source inspection data collected by a target UAV during inspection of a target inspection area according to a preset inspection plan, and performing multi-dimensional preprocessing on the multi-source inspection data to obtain spatiotemporally aligned data; using the spatiotemporally aligned data to perform three-dimensional scene modeling of the target inspection area to obtain a three-dimensional scene model and a dynamic environment model; acquiring inspection operation instructions, and allocating inspection tasks with multiple flight constraints to the dynamic environment model based on the inspection operation instructions to obtain a task execution plan; performing global path planning on the task execution plan based on the three-dimensional scene model to obtain an initial inspection path, and performing local spatiotemporal collaborative path optimization on the initial inspection path to obtain a target inspection path for the target UAV; remotely controlling the target UAV to perform optimized path inspection based on the target inspection path to obtain optimized path inspection data, performing anomaly detection on the optimized path inspection data to obtain anomaly detection results, and generating an inspection report for the target inspection area based on the anomaly detection results.

[0007] Further, the step of using the spatiotemporal aligned data to perform 3D scene modeling of the target inspection area to obtain a 3D scene model and a dynamic environment model includes: segmenting the spatiotemporal aligned data to obtain a point cloud sequence and a multimodal image sequence; analyzing the point cloud sequence and the multimodal image sequence to obtain an initial pose estimate, pixel-level semantic labels, multi-dimensional environmental information, and a time-varying environmental factor field; adjusting the reprojection error of the initial pose estimate to obtain an accurate camera trajectory; and based on the accurate camera trajectory, performing registration, fusion, and triangulation on the point cloud sequence. An initial 3D scene framework is obtained; the pixel-level semantic tags are projected onto the initial 3D scene framework to obtain inspection scene elements with category tags, and the inspection scene elements are segmented and parametrically modeled to obtain the 3D scene model; the 3D scene model is encoded using multi-resolution octree to obtain 3D scene storage data, and the multi-dimensional environmental information and the 3D scene storage data are superimposed to obtain the inspection environment representation; the inspection environment representation and the time-varying environmental factor field are integrated with spatiotemporal features to obtain the dynamic environment model.

[0008] Further, the steps of analyzing the point cloud sequence and multimodal image sequence to obtain initial pose estimation, pixel-level semantic labels, multidimensional environmental information, and time-varying environmental factor field include: performing point cloud feature extraction and inter-frame matching on the point cloud sequence to obtain initial pose estimation; performing semantic segmentation on the multimodal image sequence to obtain pixel-level semantic labels, and performing time-series analysis on the multispectral data, temperature distribution map, and parameter distribution in the multimodal image sequence to obtain multidimensional environmental information; extracting time-series environmental parameter sequences from the multidimensional environmental information, and performing interpolation and spatial mapping on the time-series environmental parameter sequences to obtain a continuous environmental parameter field; and performing time-series analysis on the continuous environmental parameter field to obtain a time-varying environmental factor field.

[0009] Furthermore, the step of allocating inspection tasks with multiple flight constraints to the dynamic environment model based on the inspection operation instructions to obtain a task execution plan includes: extracting task features corresponding to the inspection operation from the dynamic environment model based on the inspection operation instructions to obtain a multi-dimensional task feature vector; performing cluster analysis and task priority ranking on the multi-dimensional task feature vector to obtain the task execution order of the optimized path inspection; matching the task execution order with the UAV performance parameters based on the UAV performance parameters of the target UAV to obtain an initial task allocation plan; and performing multi-objective optimization and task decision analysis on the initial task allocation plan to obtain the task execution plan.

[0010] Furthermore, based on the three-dimensional scene model, the step of performing global path planning for the task execution scheme to obtain the initial inspection path includes: rasterizing the three-dimensional scene in the three-dimensional scene model to obtain a multi-resolution probabilistic map of the target inspection area, and performing topological analysis on the multi-resolution probabilistic map to obtain a critical path node network; performing heuristic search of the initial inspection points on the critical path node network to obtain an initial global path, and performing dynamic constraint checks and path optimization on the initial global path to obtain a smooth inspection path that satisfies the dynamic constraints; performing time-varying wind field compensation on the smooth inspection path to obtain a wind field adaptive path for the target inspection area, and performing energy consumption assessment and multi-objective evaluation on the wind field adaptive path to obtain the initial inspection path.

[0011] Further, the step of performing local spatiotemporal collaborative path optimization on the initial inspection path to obtain the target inspection path of the target UAV includes: performing spatiotemporal decomposition on the initial inspection path to obtain multiple local path segments and corresponding road segment time windows, and extracting the environmental dynamic features within each road segment time window to obtain a time-varying environmental parameter set; based on the time-varying environmental parameter set, performing dynamic obstacle trajectory prediction on each local path segment to obtain a set of expected conflict points, and performing spatiotemporal density analysis on the set of expected conflict points to obtain high-risk areas and high-risk time periods, wherein the high-risk area refers to an area where the regional risk value is higher than a preset risk threshold, and the high-risk time period refers to a time period where the time period risk value is higher than the preset risk threshold; based on the high-risk area and the high-risk time period, performing multi-UAV collaboration on the target UAV. The process involves perception planning to obtain an initial distribution of observation points, followed by information gain evaluation to obtain an optimized sequence of observation locations. Path fusion is then performed on the observation location sequence and the local path segments to obtain an initial gain path. Dynamic constraints and time window compatibility extensions are applied to the initial gain path to obtain a set of candidate local paths that meet time constraints. Multiple local parameter optimization is performed on the candidate local path set to obtain an optimal local path. Using the optimal local path, a dynamic artificial potential field that satisfies preset time factors is constructed to obtain a time-varying obstacle avoidance force field. Multi-UAV spatiotemporal collaborative optimization is performed on the time-varying obstacle avoidance force field to obtain a collaborative obstacle avoidance strategy that satisfies the time windows of multiple UAVs and road segments. Finally, the optimal local path, the collaborative obstacle avoidance strategy, and the road segment time window are integrated to obtain the target inspection path for the target UAV.

[0012] Furthermore, the step of remotely controlling the target UAV to perform optimized path inspection based on the target inspection path to obtain optimized path inspection data includes: performing remote global inspection of the target UAV based on the global inspection path in the target inspection path to obtain global inspection data; performing remote local inspection of the inspection local area corresponding to the local inspection path based on the local inspection path in the target inspection path to obtain local inspection data; performing spatiotemporal registration of the global inspection data and the local inspection data to obtain a multi-scale inspection dataset, and performing multimodal data fusion of the multi-scale inspection dataset to obtain optimized path inspection data with comprehensive feature representation.

[0013] According to another aspect of the present invention, a dynamic inspection system based on a drone is also provided, comprising: an acquisition unit, configured to acquire multi-source inspection data collected by a target drone during inspection of a target inspection area according to a preset inspection plan, and to perform multi-dimensional preprocessing on the multi-source inspection data to obtain spatiotemporally aligned data; a modeling unit, configured to use the spatiotemporally aligned data to perform three-dimensional scene modeling of the target inspection area to obtain a three-dimensional scene model and a dynamic environment model; an allocation unit, configured to acquire inspection operation instructions, and to allocate inspection tasks with multiple flight constraints to the dynamic environment model based on the inspection operation instructions to obtain a task execution plan; an optimization unit, configured to perform global path planning on the task execution plan based on the three-dimensional scene model to obtain an initial inspection path, and to perform local spatiotemporal collaborative path optimization on the initial inspection path to obtain a target inspection path for the target drone; and an inspection unit, configured to remotely control the target drone to perform optimized path inspection based on the target inspection path, obtain optimized path inspection data, perform anomaly detection on the optimized path inspection data to obtain anomaly detection results, and generate an inspection report for the target inspection area based on the anomaly detection results.

[0014] Further, the modeling unit includes: a first segmentation module, used to segment the spatiotemporally aligned data to obtain a point cloud sequence and a multimodal image sequence; a first analysis module, used to analyze the point cloud sequence and the multimodal image sequence to obtain initial pose estimation, pixel-level semantic labels, multi-dimensional environmental information, and a time-varying environmental factor field; a first adjustment module, used to adjust the reprojection error of the initial pose estimation to obtain an accurate camera trajectory, and based on the accurate camera trajectory, to perform registration fusion and triangulation on the point cloud sequence to obtain an initial 3D scene framework; and a first modeling module, used for... The pixel-level semantic tags are projected onto the initial 3D scene framework to obtain inspection scene elements with category tags. The inspection scene elements are then segmented into instances and parametrically modeled to obtain the 3D scene model. A first encoding module performs multi-resolution octree encoding on the 3D scene model to obtain 3D scene storage data. The multi-dimensional environmental information and the 3D scene storage data are then superimposed to obtain an inspection environment representation. A first integration module integrates the inspection environment representation and the time-varying environmental factor field to obtain the dynamic environment model.

[0015] Further, the first analysis module includes: a first matching submodule, used to perform point cloud feature extraction and inter-frame matching on the point cloud sequence to obtain an initial pose estimate; a first analysis submodule, used to perform semantic segmentation on the multimodal image sequence to obtain pixel-level semantic labels, and to perform time-series analysis on the multispectral data, temperature distribution map, and parameter distribution in the multimodal image sequence to obtain multidimensional environmental information; and a first extraction submodule, used to extract a time-series environmental parameter sequence from the multidimensional environmental information, and to perform interpolation and spatial mapping on the time-series environmental parameter sequence to obtain a continuous environmental parameter field, and to perform time-series analysis on the continuous environmental parameter field to obtain a time-varying environmental factor field.

[0016] Further, the allocation unit includes: a first extraction module, used to extract task features corresponding to the inspection operation from the dynamic environment model based on the inspection operation instruction, to obtain a multi-dimensional task feature vector; a first sorting module, used to perform cluster analysis and task priority sorting on the multi-dimensional task feature vector to obtain the task execution order of the optimized path inspection; a first matching module, used to perform drone performance parameter matching on the task execution order based on the drone performance parameters of the target drone, to obtain an initial task allocation scheme; and a first optimization module, used to perform multi-objective optimization and task decision analysis on the initial task allocation scheme to obtain the task execution scheme.

[0017] Further, the optimization unit includes: a second analysis module, used to rasterize the 3D scene in the 3D scene model to obtain a multi-resolution probabilistic map of the target inspection area, and to perform topological analysis on the multi-resolution probabilistic map to obtain a critical path node network; a first search module, used to perform a heuristic search for initial inspection points on the critical path node network to obtain an initial global path, and to perform dynamic constraint checks and path optimization on the initial global path to obtain a smooth inspection path that satisfies the dynamic constraints; and a first evaluation module, used to perform time-varying wind field compensation on the smooth inspection path to obtain a wind field adaptive path for the target inspection area, and to perform energy consumption evaluation and multi-objective evaluation on the wind field adaptive path to obtain an initial inspection path.

[0018] Furthermore, the optimization unit further includes: a first decomposition module, used to perform spatiotemporal decomposition on the initial inspection path to obtain multiple local path segments and corresponding road segment time windows, and extract the environmental dynamic features within each road segment time window to obtain a time-varying environmental parameter set; a first prediction module, used to perform dynamic obstacle trajectory prediction on each local path segment based on the time-varying environmental parameter set to obtain a set of expected conflict points, and perform spatiotemporal density analysis on the set of expected conflict points to obtain high-risk areas and high-risk time periods, wherein the high-risk area refers to an area with a regional risk value higher than a preset risk threshold, and the high-risk time period refers to a time period with a time period risk value higher than the preset risk threshold; a first planning module, used to perform multi-drone collaborative perception planning on the target UAV based on the high-risk area and the high-risk time period to obtain an initial observation point distribution, and to... The initial observation point distribution is evaluated for information gain to obtain an optimized observation position sequence. A first fusion module is used to fuse the observation position sequence and the local path segments to obtain an initial gain path, and to extend the initial gain path with dynamic constraints and time window compatibility to obtain a set of candidate local paths that meet the time constraints. A first construction module is used to optimize the candidate local path set with multiple local parameters to obtain an optimal local path, and to use the optimal local path to construct a dynamic artificial potential field that meets the preset time factor to obtain a time-varying obstacle avoidance force field. A second integration module is used to perform multi-UAV spatiotemporal collaborative optimization on the time-varying obstacle avoidance force field to obtain a collaborative obstacle avoidance strategy that meets the time windows of multiple UAVs and road segments, and to integrate the optimal local path, the collaborative obstacle avoidance strategy and the road segment time window to obtain the target inspection path of the target UAV.

[0019] Furthermore, the inspection unit includes: a first inspection module, used to perform remote global inspection of the target UAV based on the global inspection path in the target inspection path, to obtain global inspection data; a second inspection module, used to perform remote local inspection of the local inspection area corresponding to the local inspection path using the target UAV based on the local inspection path in the target inspection path, to obtain local inspection data; and a second fusion module, used to perform spatiotemporal registration of the global inspection data and the local inspection data to obtain a multi-scale inspection dataset, and to perform multimodal data fusion of the multi-scale inspection dataset to obtain optimized path inspection data with comprehensive feature representation.

[0020] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform any of the above-described dynamic inspection methods based on unmanned aerial vehicles.

[0021] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement any of the above-described dynamic inspection methods based on unmanned aerial vehicles.

[0022] In this application, the following steps are performed: First, multi-source inspection data is collected when the target UAV performs inspection operations on the target inspection area according to a preset inspection plan. This multi-source inspection data undergoes multi-dimensional preprocessing to obtain spatiotemporally aligned data. Then, a 3D scene model of the target inspection area is created using the spatiotemporally aligned data, resulting in a 3D scene model and a dynamic environment model. Next, inspection operation instructions are obtained, and inspection tasks with multiple flight constraints are allocated to the dynamic environment model based on these instructions, resulting in a task execution plan. Based on the 3D scene model, global path planning is performed on the task execution plan to obtain an initial inspection path. Then, local spatiotemporal collaborative path optimization is performed on the initial inspection path to obtain the target UAV's target inspection path. Finally, based on the target inspection path, the target UAV is remotely controlled to perform optimized path inspection, obtaining optimized path inspection data. Anomaly detection is performed on the optimized path inspection data to obtain anomaly detection results. Finally, an inspection report for the target inspection area is generated based on the anomaly detection results.

[0023] This application proposes a secondary inspection mechanism. A primary inspection is performed using a pre-planned route. The primary inspection data is then fully utilized for modeling, constructing a 3D scene model and a dynamic environment model. Based on the dynamic environment model, inspection tasks with multiple flight constraints are allocated. Combined with the 3D scene model, dynamic path planning is performed to achieve precise path planning. A secondary inspection is then executed based on the planned path to identify anomalies in the target area. This achieves the technical effect of improving the accuracy of inspection results and effectively identifying anomalies. Furthermore, it solves the technical problem in related technologies where inspection methods based on historical data for path planning have poor adaptability and low accuracy. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0025] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a dynamic inspection method based on unmanned aerial vehicles is shown.

[0026] Figure 2This is a flowchart of an optional dynamic inspection method based on unmanned aerial vehicles according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of an optional UAV-based dynamic inspection system according to an embodiment of the present invention;

[0028] Figure 4 This is a hardware structure block diagram of an optional electronic device (or mobile device) for implementing a dynamic inspection method based on unmanned aerial vehicles according to an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] To facilitate understanding of the present invention by those skilled in the art, some terms or nouns involved in the various embodiments of the present invention are explained below:

[0032] Global Positioning System, or GPS for short.

[0033] Micro-Electro-Mechanical Systems, or MEMS for short.

[0034] Multi-resolution octree coding is an efficient algorithm for the structured representation and storage of three-dimensional spatial data.

[0035] Mask R-CNN is an object detection and instance segmentation algorithm based on deep learning. It is specifically designed for object detection and segmentation, capable of identifying multiple objects in an image and generating accurate pixel-level segmentation masks for each object.

[0036] A-Star algorithm, a pathfinding algorithm, is a heuristic search algorithm for finding the shortest path in a graph. It is widely used in path planning, game AI, network routing, and other problems that require finding optimal solutions.

[0037] Scale-Invariant Feature Transform (SIFT) is a computer vision algorithm used to extract and describe keypoints from images that are scale-invariant and rotation-invariant.

[0038] Random Sample Consensus (RANSAC) is an iterative algorithm for estimating parametric models, especially when the dataset may contain outliers. It can find the optimal model parameters from a set of observations.

[0039] One-Class Support Vector Machine (SVM) is a supervised learning algorithm used in anomaly detection. It can learn and build a boundary model of normal categories when only normal sample data is available, thereby distinguishing between normal and abnormal data.

[0040] It should be noted that the UAV-based dynamic inspection method and apparatus in this application can be used in the field of software and platform technology for secondary inspection and anomaly detection using UAVs, and can also be used in any field other than the field of software and platform technology for secondary inspection and anomaly detection using UAVs. This application does not limit the application field of the UAV-based dynamic inspection method and apparatus.

[0041] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding access points are provided for users to choose to authorize or refuse. For example, interfaces are set up between this system and relevant users or organizations, providing users with corresponding access points to choose to agree to or refuse automated decision-making results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0042] The following embodiments of the present invention can be applied to various UAV-based dynamic inspection systems / applications / equipment. The present invention proposes a secondary inspection mechanism, which fully utilizes primary inspection data for modeling, constructing a 3D scene model and a dynamic environment model, and performing multi-constraint dynamic task allocation to obtain a task execution plan for the secondary inspection. Then, based on the high-precision 3D scene model, global path planning and local obstacle avoidance path optimization are performed on the task execution plan to obtain the final inspection path for the target UAV during the secondary inspection. Based on the final inspection path, the target UAV can be remotely controlled to perform a secondary anomaly inspection, identifying anomalies in the target area and improving the accuracy of the inspection results.

[0043] The present invention will now be described in detail with reference to various embodiments.

[0044] Example 1

[0045] According to an embodiment of the present invention, an embodiment of a dynamic inspection method based on unmanned aerial vehicles is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0046] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a drone-based dynamic inspection method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0047] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0048] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the UAV-based dynamic inspection method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned UAV-based dynamic inspection method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0049] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0050] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0051] Under the aforementioned operating environment, this application provides the following: Figure 2 The method shown is a dynamic inspection method based on drones. The main body of this method is a dynamic inspection system based on drones.

[0052] Figure 2 This is a flowchart of an optional dynamic inspection method based on unmanned aerial vehicles (UAVs) according to an embodiment of the present invention, such as... Figure 2As shown, the method includes the following steps:

[0053] Step S201: Obtain multi-source inspection data collected by the target UAV when it performs inspection operations on the target inspection area according to the preset inspection plan, and perform multi-dimensional preprocessing on the multi-source inspection data to obtain spatiotemporally aligned data.

[0054] In this embodiment, the UAV-based dynamic inspection system is applied to a UAV inspection system composed of at least one UAV, at least one UAV payload, and at least one UAV airport. Inspecting a target inspection area according to a preset inspection plan refers to the UAV performing an inspection operation according to the pre-set plan. The preset inspection plan can be determined based on relevant information from historical data. Upon completion of an inspection operation, multi-source inspection data of the target inspection area is collected. This multi-source raw inspection data can include navigation and positioning data, inertial measurement data, inspection images, multispectral images, inspection infrared images, and environmental parameter data, etc.

[0055] In one optional embodiment, if the data collected by the drone is subject to environmental interference, the collected multi-source inspection data first needs to be preprocessed to ensure data accuracy. The process of multi-dimensional preprocessing of multi-source inspection data includes:

[0056] First, adaptive Kalman filtering is applied to the navigation and positioning data and inertial measurement data in the multi-source inspection data to obtain preliminary fused position information. Then, inspection map matching is performed on the preliminary fused position information to obtain corrected high-precision position data.

[0057] Secondly, distortion correction and noise reduction are performed on the inspection images in the multi-source inspection data to obtain the corrected visible light image. Then, stereo vision representation is performed on the corrected visible light image to obtain the initial depth map. Point cloud generation is performed on the initial depth map to obtain the original point cloud data.

[0058] Subsequently, spectral and atmospheric corrections were performed on the multispectral images in the multi-source inspection data to obtain corrected multispectral data. Non-uniformity correction and temperature calibration were performed on the inspection infrared images in the multi-source inspection data to obtain the temperature distribution map of the initial inspection. Noise filtering was performed on the environmental parameter data. Time synchronization and spatial registration were performed on the corrected visible light images, multispectral data, temperature distribution map and environmental parameter data to obtain an aligned multimodal dataset.

[0059] Data fusion is performed on the original point cloud data and the multimodal dataset to obtain attribute-enhanced inspection point cloud data. Then, the inspection point cloud data is denoised and thinned to obtain optimized inspection point cloud data.

[0060] Finally, the optimized inspection point cloud data and high-precision location data are spatiotemporally aligned to obtain cleaned spatiotemporally aligned data.

[0061] In practical applications, when the target UAV conducts inspections of the target inspection area according to a preset inspection plan, it acquires multi-source inspection data through relevant UAV sensors. For example, it acquires navigation and positioning data through a dual-frequency GPS / BeiDou combined navigation module, acquires inertial measurement data corresponding to acceleration, angular velocity, and magnetic field data using a 9-axis MEMS inertial measurement unit, acquires inspection images using a 4K ultra-high-definition visible light camera, acquires image data in the 400-1000nm band using an 8-channel multispectral camera, acquires one set of multispectral images per second, acquires inspection infrared images using a miniaturized long-wave infrared thermal imager, acquires environmental parameters using a multi-sensor array, including environmental parameter data such as temperature, humidity, air pressure, wind speed, and various gas concentrations, and records a comprehensive log of the UAV system, including UAV flight parameter data such as battery management system data, motor speed, flight control status, and communication quality.

[0062] Then, adaptive Kalman filtering is applied to the navigation and positioning data and inertial measurement data to obtain preliminary fused position information. Particle filtering algorithm is then used to perform map matching on the preliminary fused position information, thereby matching the fused position information with known maps to further improve position accuracy and obtain corrected high-precision position data.

[0063] Then, the camera intrinsic parameters and distortion coefficients are obtained through calibration. Then, the distortion model is applied to each pixel in the inspection image for correction. In addition, a non-local mean algorithm is used to first search for image blocks similar to the current pixel in the inspection image. Then, the inspection image blocks are weighted and averaged according to the similarity to obtain the denoised pixel value. By repeating this process for each pixel in the image, the corrected visible light image is finally obtained.

[0064] A semi-global matching algorithm is used to calculate the pixel matching cost in the visible light image. Then, the cost is aggregated in multiple directions. Finally, dynamic programming is used to find the globally optimal disparity map, resulting in a high-quality initial depth map. Furthermore, by using a back-projection method, the coordinates of each pixel in the depth map are calculated in 3D space based on its image coordinates and depth value, combined with the camera intrinsic parameter matrix. This process requires traversing every pixel in the depth map and generating a corresponding 3D point for each valid depth value, ultimately forming a complete original point cloud data.

[0065] Then, spectral and atmospheric corrections are performed on each band of the multispectral image to obtain corrected multispectral data. Uniform field images at two different temperatures are acquired from the inspection infrared image. The gain and offset coefficients of each pixel are calculated, and these coefficients are applied to each pixel of the original image for correction. Radiance data of targets with known temperatures are collected to establish a mapping relationship between radiance and temperature (usually based on Planck's radiation law). For each pixel in the corrected infrared image, the corresponding temperature value is calculated based on its radiance value, thus obtaining a temperature distribution map for one inspection. Wavelet transform is used to filter noise from various parameters (such as temperature, humidity, wind speed, etc.) in the environmental parameter data. A dynamic time warping algorithm is used to find the optimal matching path between the corrected visible light image, multispectral data, temperature distribution map, environmental parameters, and time series, thereby achieving time alignment.

[0066] By optimizing the transformation parameters of the time-aligned data, the mutual information between different modal data is maximized, thereby achieving spatial alignment and obtaining an aligned multimodal dataset. Then, Gaussian process regression is used to predict attributes of the original point cloud data and the multimodal dataset using a trained Gaussian process model. This requires building a model and performing prediction fusion for each attribute to be fused (such as spectral information, temperature, etc.) to obtain attribute-enhanced inspection point cloud data. A statistical outlier removal algorithm is used to calculate the average distance from each point in the inspection point cloud data to its nearest neighbor. Then, a threshold is determined based on the statistical distribution of these distances. Finally, points exceeding the threshold are removed. The point cloud data after noise reduction is divided into equal-sized voxels using a voxel grid method. The average value or center point of all points in each voxel is used to represent the points in that voxel to obtain optimized inspection point cloud data.

[0067] Then, using the obtained high-precision location data, coordinate transformation and time interpolation are performed on the point cloud data to align it with the global reference system, ultimately obtaining cleaned spatiotemporally aligned data.

[0068] Step S202: Use spatiotemporal alignment data to perform 3D scene modeling of the target inspection area to obtain a 3D scene model and a dynamic environment model.

[0069] In this embodiment, spatiotemporal aligned data obtained from a single inspection is used for 3D modeling to construct a 3D scene model and a dynamic environment model of the target area. The 3D scene model is a high-precision 3D scene representation of the target area, incorporating both static geographical scenes and dynamic activity scenes of the target area. The dynamic environment model refers to an environment representation system that can be updated and adjusted in real time, capable of capturing and reflecting changes in the environment, and continuously optimizing its own model based on new observation data. It can not only accurately represent the current environmental state, but also reflect the dynamic changes of the environment, predict future trends, and quantify uncertainties.

[0070] Furthermore, the steps for modeling the target inspection area in three dimensions using spatiotemporally aligned data to obtain a three-dimensional scene model and a dynamic environment model include: segmenting the spatiotemporally aligned data to obtain point cloud sequences and multimodal image sequences; analyzing the point cloud sequences and multimodal image sequences to obtain initial pose estimation, pixel-level semantic labels, multi-dimensional environmental information, and time-varying environmental factor fields; adjusting the reprojection error of the initial pose estimation to obtain accurate camera trajectories, and based on the accurate camera trajectories, registering and fusion and triangulating the point cloud sequences to obtain an initial three-dimensional scene framework; projecting pixel-level semantic labels onto the initial three-dimensional scene framework to obtain inspection scene elements with category labels, and performing instance segmentation and parametric modeling on the inspection scene elements to obtain a three-dimensional scene model; performing multi-resolution octree encoding on the three-dimensional scene model to obtain three-dimensional scene storage data, and superimposing features of the multi-dimensional environmental information and the three-dimensional scene storage data to obtain an inspection environment representation; and integrating the spatiotemporal features of the inspection environment representation and the time-varying environmental factor field to obtain a dynamic environment model.

[0071] Further, the steps for analyzing point cloud sequences and multimodal image sequences to obtain initial pose estimation, pixel-level semantic labels, multidimensional environmental information, and time-varying environmental factor fields include: extracting point cloud features and performing inter-frame matching on the point cloud sequences to obtain initial pose estimation; performing semantic segmentation on the multimodal image sequences to obtain pixel-level semantic labels, and performing time-series analysis on the multispectral data, temperature distribution maps, and parameter distribution patterns in the multimodal image sequences to obtain multidimensional environmental information; extracting time-series environmental parameter sequences from the multidimensional environmental information, and performing interpolation and spatial mapping on the time-series environmental parameter sequences to obtain a continuous environmental parameter field; and performing time-series analysis on the continuous environmental parameter field to obtain a time-varying environmental factor field.

[0072] When performing 3D scene modeling, the spatiotemporally aligned data is first segmented by timestamps and data types to obtain point cloud sequences and multimodal image sequences. A point cloud sequence refers to a collection of multiple point cloud data acquired consecutively over a period of time, with each point cloud dataset consisting of the positional information of a large number of 3D points in space. A multimodal image sequence refers to a series of images captured by different types of image sensors, which may include visible light images, infrared images, depth images, radar images, etc. Multimodal images provide environmental information from different perspectives or with varying physical properties, enabling a more comprehensive description of the scene.

[0073] For point cloud sequences, 3D feature descriptors are used to extract point cloud features in order to effectively capture the local geometric features of the point cloud, and the ICP (Iterative Nearest Point) algorithm is used for inter-frame matching to obtain the initial pose estimate. The initial pose estimate refers to the preliminary estimate of the starting position and attitude of the operating system (such as a UAV) in three-dimensional space when the operating system starts running.

[0074] For multimodal image sequences, dilated convolution and encoder-decoder structures from deep learning algorithms are used to perform semantic segmentation on the multimodal image sequences in order to effectively capture multi-scale contextual information and obtain pixel-level semantic labels to identify and distinguish different objects or regions in the image. For example, in a city landscape image, semantic segmentation can label different objects such as sky, buildings, trees, roads, and pedestrians separately, and give each pixel a corresponding label, such as "sky", "buildings", "roads", etc.

[0075] Time series analysis techniques, such as ARIMA (AutoRegressive Integrated Moving Average), are used to analyze the multispectral data, temperature distribution maps, and parameter distribution in multimodal image sequences to capture the changing trends, seasonal variations, and periodic changes of various data, thus obtaining a multi-dimensional environmental information layer. Kriging interpolation is used to interpolate the time-series environmental parameter sequences, and dynamic time warping algorithms are applied to analyze the similarity and change patterns of the time series, obtaining the time-varying environmental factor field during the initial inspection of the target UAV. Furthermore, the reprojection error of the initial pose estimation is adjusted. The reprojection error refers to projecting the feature points detected in the current frame onto the reference frame based on the current pose estimation and the pose information of the reference frame, and then comparing the difference with the matching points found in the reference frame. This error is usually expressed as pixel distance or point cloud spatial distance, reflecting the accuracy of the current estimated pose. Simultaneously, by optimizing the camera pose and the position of 3D points in the initial pose estimation, an accurate camera trajectory is obtained. Based on the obtained accurate camera trajectory, the point cloud sequence is registered and fused using the probabilistic ICP algorithm, and the point cloud reconstruction is performed by solving the Poisson equation using the Poisson reconstruction algorithm to process noise and incomplete point cloud data, thereby obtaining an initial 3D environment framework. Then, pixel-level semantic labels are projected onto the initial 3D environment framework through back projection and nearest neighbor search to obtain inspection scene elements with category labels. Deep learning models such as Mask R-CNN are used to perform instance segmentation of the inspection scene elements, and the random sampling consensus algorithm is used to fit geometric primitives for parametric modeling, resulting in a lightweight 3D scene representation.

[0076] To improve storage and access efficiency, multi-resolution octree encoding is applied to the 3D scene representation to obtain 3D scene storage data. For heterogeneous data with spatial relationships, a graph convolutional network is used to overlay features of the multi-dimensional environmental information layer and the 3D scene storage data to obtain an inspection scene representation rich in multi-dimensional information. Furthermore, a spatiotemporal graph convolutional network (ST-GCN) is used to model spatiotemporal relationships. The integration formula of ST-GCN is expressed as:

[0077] ;

[0078] in, It is the first Layer node characteristics, It is an adjacency matrix. It is the identity matrix. It is a degree matrix. It is a learnable weight matrix. This is an activation function used to integrate the spatiotemporal features of the inspection scene representation and the time-varying environmental factor field using ST-GCN, while capturing the dependencies in both spatial and temporal dimensions to obtain a high-precision 3D scene model and a dynamic environment model. The dynamic environment model not only accurately represents the physical structure of the inspection area but also reflects the spatiotemporal distribution and changing trends of environmental parameters. For example, this model can be used to query the temperature distribution of a building at a specific point in time or predict the environmental conditions at a future point in time, providing strong data support for subsequent tasks such as anomaly detection and predictive maintenance.

[0079] Step S203: Obtain inspection operation instructions, and allocate inspection tasks with multiple flight constraints to the dynamic environment model based on the inspection operation instructions to obtain a task execution plan.

[0080] In this embodiment, the inspection operation instructions refer to the operation instructions generated by the target inspector for abnormal inspection points or a customized operation inspection path including multiple inspection points when optimizing the inspection path. Multiple flight constraints may include: UAV performance constraints: maximum flight time, maximum flight distance, carrying capacity, energy consumption rate, etc.; dynamic constraints: kinematic and dynamic limitations of the UAV; time window constraints: time limits for task execution; environmental constraints: environmental factors such as wind field and obstacles; and task priority constraints: priorities determined based on task urgency, potential risks, and expected benefits. By integrating multiple flight constraints into a dynamic environment model, dynamic allocation of multiple constraints for inspection tasks is performed, thereby obtaining a task execution plan. Inspection tasks may include: detailed inspection of specific areas; secondary confirmation of anomalies; in-depth investigation of abnormal situations; and specific inspection operations based on the inspection operation instructions.

[0081] Furthermore, based on the inspection operation instructions, the steps for allocating inspection tasks with multiple flight constraints to the dynamic environment model to obtain a task execution plan include: extracting task features corresponding to the inspection operations from the dynamic environment model based on the inspection operation instructions to obtain a multi-dimensional task feature vector; performing cluster analysis and task priority ranking on the multi-dimensional task feature vector to obtain the task execution order of the optimized path inspection; matching the task execution order with the UAV performance parameters based on the UAV performance parameters of the target UAV to obtain an initial task allocation plan; and performing multi-objective optimization and task decision analysis on the initial task allocation plan to obtain the task execution plan.

[0082] In practical applications, the process begins by obtaining remote inspection instructions from the target inspector. These instructions may include detailed inspections of specific areas, secondary confirmation of anomalies, or in-depth investigations of newly discovered problems. Based on these instructions, relevant task features are extracted from a previously constructed dynamic environment model. These features may include the terrain complexity of the target area, obstacle distribution, the degree of anomaly in environmental parameters (such as temperature, humidity, and light), and differences from previous inspection results. These features are organized into a multi-dimensional task feature vector, with each dimension representing a specific task attribute or environmental characteristic. Clustering algorithms are then used to perform cluster analysis on the multi-dimensional task feature vector to group similar tasks or areas, facilitating subsequent task planning and resource allocation. Based on the clustering, tasks can be prioritized using methods such as the analytic hierarchy process (AHP). Prioritization may consider multiple factors, such as task urgency, potential risks, and expected benefits, to obtain an optimized path for the execution order of inspection tasks. This order considers both the similarity and importance of the tasks. Then, based on the performance parameters of the target UAV (including maximum flight time, maximum flight distance, carrying capacity, energy consumption rate, etc.), a heuristic algorithm or constraint satisfaction problem solver is used to match the UAV performance parameters to the task execution order. Simultaneously, multiple constraints such as dynamic constraints, time window constraints, environmental constraints, and task priority constraints are combined to find a task allocation scheme that meets the conditions, resulting in an initial task allocation scheme. This scheme specifies which tasks should be executed by which UAV and the approximate execution order. A non-dominated sorting genetic algorithm is then used to perform multi-objective optimization on each initial task allocation scheme. The optimization objectives include minimizing total flight time, maximizing task completion rate, minimizing energy consumption, and maximizing inspection coverage. Furthermore, techniques such as fuzzy logic, Bayesian networks, or Monte Carlo simulation are used during multi-objective optimization to evaluate the risks and benefits of different schemes, ultimately yielding an optimized task execution scheme for the initial task allocation.

[0083] Step S204: Based on the 3D scene model, perform global path planning for the task execution plan to obtain the initial inspection path, and perform local spatiotemporal collaborative path optimization on the initial inspection path to obtain the target inspection path of the target UAV.

[0084] In step S204 above, a high-precision 3D scene model is used as the basis for path planning. By analyzing the geographical information, obstacle distribution, and terrain features in the model, a global path planning algorithm is employed to generate an initial inspection path connecting the inspection start and end points. This path is designed to maximize inspection efficiency while ensuring the UAV can perform its mission within a safe range. Local spatiotemporal collaborative path optimization is then performed on the generated initial inspection path. This step focuses on specific areas along the path, especially inspection points near obstacles requiring detailed inspection or areas with frequently changing dynamic environments. By considering factors such as the UAV's real-time status, the dynamic changes of obstacles, and the specific requirements of the inspection task, the system uses local obstacle avoidance algorithms, time series analysis, or collaborative optimization strategies to fine-tune the UAV's flight path, altitude, and speed within this area to address the uncertainties of the local environment and improve the flexibility and response speed of the inspection.

[0085] Furthermore, based on the 3D scene model, the steps for global path planning of the task execution scheme to obtain the initial inspection path include: rasterizing the 3D scene in the 3D scene model to obtain a multi-resolution probability map of the target inspection area, and performing topology analysis on the multi-resolution probability map to obtain a critical path node network; performing heuristic search of the initial inspection points on the critical path node network to obtain the initial global path, and performing dynamic constraint checks and path optimization on the initial global path to obtain a smooth inspection path that satisfies the dynamic constraints; performing time-varying wind field compensation on the smooth inspection path to obtain a wind field adaptive path for the target inspection area, and performing energy consumption assessment and multi-objective evaluation on the wind field adaptive path to obtain the initial inspection path.

[0086] In practical applications, an adaptive octree structure is first used to rasterize the 3D scene in the high-precision 3D scene model. The resolution of the raster is dynamically adjusted according to the complexity of the scene, and the continuous 3D space is discretized into multiple raster cells. Each raster cell contains information such as occupancy probability and terrain features, thus obtaining a multi-resolution probabilistic map of the target inspection area. A median transformation algorithm is then used to perform topological analysis on the multi-resolution probabilistic map to identify key passages, intersections, and open spaces in the scene, forming a simplified path node network, thus obtaining the critical path node network, which greatly reduces the search space for subsequent path planning. Then, an improved A-star pathfinding algorithm is used to perform a heuristic search for initial inspection points in the critical path node network. This allows skipping some intermediate nodes during the search process and directly considering key points that may change the path, thus significantly reducing the number of nodes searched and obtaining a preliminary global path. Finally, a model predictive control method is used to perform dynamic constraint checks and path optimization on the preliminary global path based on the current state and the prediction model. The objective function of the model predictive control method is: , It is the objective function. It is the state of the drone at time k. It is a reference trajectory. The control input is Q, and the weight matrices are R. By minimizing the objective function, a smooth inspection path that satisfies dynamic constraints is obtained. Then, by introducing a wind field model and modifying the UAV's motion equations, time-varying wind field compensation is performed on this smooth path; that is, by assuming a wind field prediction model... Let x represent the wind speed at position x and time t. Then the equation of motion for the drone can be expressed as:

[0087] ;

[0088] in, The dynamic model of the UAV under windless conditions is represented. By solving the above motion equation, an adaptive path considering the influence of wind field is obtained. Then, the energy consumption and multi-objective evaluation of the wind field adaptive path are carried out. Specifically, the energy consumption is evaluated by simulation calculation based on the UAV dynamic model and wind field data. The multi-objective optimization algorithm (such as non-dominated sorting genetic algorithm II) is used to evaluate multiple objectives (minimize path length, minimize flight time, minimize energy consumption, maximize information acquisition and task coverage, etc.) to obtain the initial inspection path.

[0089] Furthermore, the steps for optimizing the initial inspection path using local spatiotemporal collaborative paths to obtain the target inspection path for the target UAV include: spatiotemporally decomposing the initial inspection path to obtain multiple local path segments and corresponding segment time windows, and extracting the dynamic environmental features within each segment time window to obtain a time-varying environmental parameter set; based on the time-varying environmental parameter set, predicting the dynamic obstacle trajectory of each local path segment to obtain a set of expected conflict points, and performing spatiotemporal density analysis on the set of expected conflict points to obtain high-risk areas and high-risk time periods, where a high-risk area refers to an area where the regional risk value is higher than a preset risk threshold, and a high-risk time period refers to a time period where the time period risk value is higher than a preset risk threshold; and based on the high-risk areas and high-risk time periods, performing multi-UAV collaborative perception on the target UAV. The process involves planning to obtain an initial distribution of observation points, evaluating the information gain of these points to obtain an optimized sequence of observation locations, fusing the observation location sequence with local path segments to obtain an initial gain path, and extending this initial gain path with dynamic constraints and time window compatibility to obtain a set of candidate local paths that meet the time constraints. The candidate local path set is then optimized using multiple local parameters to obtain the optimal local path. Using this optimal local path, a dynamic artificial potential field that satisfies preset time factors is constructed, resulting in a time-varying obstacle avoidance force field. This time-varying obstacle avoidance force field is then optimized using a multi-UAV spatiotemporal collaboration mechanism to obtain a collaborative obstacle avoidance strategy that satisfies the time windows of multiple UAVs and road segments. Finally, the optimal local path, the collaborative obstacle avoidance strategy, and the road segment time window are integrated to obtain the target inspection path for the target UAV.

[0090] In practical applications, path segmentation algorithms, such as curvature-based segmentation methods, are used to perform spatiotemporal decomposition on the initial inspection path, dividing it into multiple local path segments and corresponding segment time windows. Then, environmental dynamic features are extracted from each segment time window to capture the time-varying characteristics of environmental parameters, forming a time-varying environmental parameter set. Based on this time-varying environmental parameter set, Kalman filtering is used to perform spatiotemporal cross-analysis of the predicted trajectory and path segments, obtaining a set of expected conflict points. Kernel density estimation is then used to perform spatiotemporal density analysis on these conflict points, identifying high-risk areas and high-risk time periods. Finally, based on the identified high-risk areas and time periods, a distributed deep Q-network enables multiple target UAVs to make collaborative decisions, generating a preliminary observation point distribution. Information gain is then evaluated using metrics such as mutual information or information entropy. To maximize information acquisition efficiency, an optimized observation position sequence is obtained. Then, using path interpolation and smoothing algorithms such as Bézier curve interpolation or spline interpolation, the optimized observation position sequence is fused with the original local path segments to obtain a preliminary gain path. Using model predictive control methods, considering both the UAV's kinematic constraints and time window limitations, a set of candidate local paths satisfying time constraints is generated. Subsequently, a multi-objective optimization algorithm is used to optimize multiple local parameters of the candidate local path set (where optimization objectives may include local path length, local energy consumption, local information gain, and local risk avoidance, etc.) to obtain the optimal local path. Based on the optimal local path, a dynamic artificial potential field satisfying preset time factors is constructed using the time-varying artificial potential field method, where the potential field function dynamically changes with time, and the potential field function can be expressed as: ,in, It is the total potential function at position x and time t. It is a gravitational potential field that changes over time. It is a repulsive potential field that changes over time. This time-varying potential field can effectively guide the UAV to avoid dynamic obstacles while moving towards the target point, thus obtaining the time-varying obstacle avoidance force field. Then, an asynchronous distributed optimization algorithm is used to perform spatiotemporal collaborative optimization of the time-varying obstacle avoidance force field for multiple UAVs, so as to coordinate the decisions of multiple UAVs in a distributed environment, generate a collaborative obstacle avoidance strategy that satisfies the constraints of multiple UAVs and time windows, and integrate the optimal local path, collaborative obstacle avoidance strategy and road segment time window through a spatiotemporal trajectory planning algorithm, so as to generate a smooth and dynamically feasible final inspection path for the target UAV while considering time constraints, thus obtaining the target inspection path.

[0091] Step S205: Based on the target inspection path, remotely control the target UAV to perform optimized path inspection, obtain optimized path inspection data, perform anomaly detection on the optimized path inspection data, obtain anomaly detection results, and generate an inspection report for the target inspection area based on the anomaly detection results.

[0092] In step S205 above, based on the optimized target inspection path, the system sends precise flight commands to the target UAV via the remote control module, controlling the UAV to perform a secondary inspection of the target area. The flight commands include detailed flight trajectory information, including but not limited to parameters such as the starting point, waypoints, destination, flight altitude, speed, and specific observation angle. After receiving these commands, the UAV will perform a secondary inspection along the optimized path, collecting optimized path inspection data such as high-definition images, multispectral images, infrared images, and environmental parameters along the route.

[0093] The optimized path inspection data collected by the drone is transmitted in real time to the anomaly analysis module. The module uses machine learning algorithms or deep learning models to detect anomalies in the data, identifying data points that are significantly different from normal inspection data. The anomaly detection process may include pixel-level analysis of images to identify abnormal objects or structures; feature extraction from multispectral and infrared images to determine if there are abnormal heat sources or material distributions; and statistical analysis of environmental parameters to detect abnormal changes in temperature, humidity, or wind speed. Once anomaly data is detected, the system associates these data points with specific location information to generate anomaly detection results.

[0094] Finally, based on the anomaly detection results, the system automatically generates an inspection report for the target inspection area. The report records the location, type, possible causes of each anomaly, and suggested countermeasures for each data point. Furthermore, the report includes overall inspection coverage, drone flight statistics, and an evaluation of the inspection task's execution efficiency, providing inspection operators and maintenance teams with a comprehensive overview of the inspection task's performance.

[0095] Furthermore, the steps for remotely controlling the target UAV to perform optimized path inspection based on the target inspection path and obtaining optimized path inspection data include: performing remote global inspection of the target UAV based on the global inspection path in the target inspection path to obtain global inspection data; performing remote local inspection of the inspection local area corresponding to the local inspection path based on the local inspection path in the target inspection path to obtain local inspection data; performing spatiotemporal registration of the global inspection data and the local inspection data to obtain a multi-scale inspection dataset, and performing multi-modal data fusion on the multi-scale inspection dataset to obtain optimized path inspection data with comprehensive feature representation.

[0096] In this embodiment, a remote global inspection of the target UAV is performed based on the global inspection path within the target inspection path. During the global inspection, the UAV flies along a predetermined path and uses various onboard sensors, such as high-resolution cameras, multispectral sensors, and lidar, to collect data over a wide area of ​​the optimized inspection area and anomalies, thereby obtaining global inspection data. Simultaneously, based on the local inspection path within the target inspection path, the target UAV performs remote, detailed inspection of the corresponding local inspection area. This process requires the UAV to conduct more detailed observations and data collection in specific areas of interest. For example, the UAV may lower its flight altitude, slow down its flight speed, or adopt specific flight modes (such as hovering or circling) to acquire higher-resolution images or more accurate sensor readings, resulting in local inspection data with higher precision and richer detail.

[0097] Spatiotemporal registration is performed on the global and local inspection data collected during secondary inspections to unify data collected at different scales and time points into a common reference coordinate system. For example, image feature points are extracted using the SIFT (Scale Invariant Feature Transform) algorithm, followed by robust matching using the RANSAC (Random Sample Consensus) algorithm. Finally, spatial transformation is used to align different datasets, resulting in a multi-scale inspection dataset that contains both global overview information and local details. Deep learning methods, such as multimodal autoencoders or cross-modal attention networks, are then used to fuse this multi-scale inspection dataset, learning the correlation and complementary information between different modalities to obtain optimized path inspection data with comprehensive feature representation. For instance, a multi-branch neural network can be designed, with each branch processing one modality of data (such as visible light images, infrared images, laser point clouds, etc.). Then, attention mechanisms or feature fusion layers are used to integrate the features of different modalities, resulting in a comprehensive and information-rich optimized path inspection data representation.

[0098] Furthermore, anomaly detection is performed on the comprehensive optimized path inspection data. For example, the Isolation Forest algorithm is used to detect outliers in numerical data, One-Class SVM is used to identify anomaly patterns, or the reconstruction error of an autoencoder is used to discover anomalous regions. For image data, a pre-trained deep convolutional neural network is used to extract features, and then anomaly detection algorithms are used to identify anomalies in the feature space. Temporal information is used, employing Long Short-Term Memory networks or temporal convolutional networks to detect anomaly patterns in time-series data. Finally, an anomaly inspection report based on an adaptive dynamic environment model is generated. The anomaly detection results require further processing and interpretation to generate an easily understandable inspection report, including the type, location, severity, possible causes, and suggested handling methods for the anomaly. Data visualization techniques, such as heatmaps or 3D rendering, can be used to visually display the spatial distribution of anomalies. In addition, during the inspection process, each UAV payload (such as sensors) can be individually controlled to achieve inspection of corresponding inspection points.

[0099] Through the above steps, multi-source inspection data collected by the target UAV during its inspection of the target inspection area according to a preset inspection plan is obtained. This multi-dimensional preprocessing of the multi-source inspection data yields spatiotemporally aligned data. The spatiotemporally aligned data is then used to create a 3D scene model of the target inspection area, resulting in a 3D scene model and a dynamic environment model. Inspection operation commands are then obtained, and based on these commands, inspection tasks with multiple flight constraints are allocated to the dynamic environment model to obtain a task execution plan. Based on the 3D scene model, global path planning is performed on the task execution plan to obtain an initial inspection path. This initial path is then optimized locally in a spatiotemporally to obtain the target UAV's target inspection path. Finally, based on the target inspection path, the target UAV is remotely controlled to perform optimized path inspection, obtaining optimized path inspection data. Anomaly detection is then performed on the optimized path inspection data to obtain anomaly detection results. Based on these anomaly detection results, an inspection report for the target inspection area is generated.

[0100] This embodiment proposes a secondary inspection mechanism. A primary inspection is performed using a pre-planned route, and the primary inspection data is fully utilized for modeling, constructing a 3D scene model and a dynamic environment model. Based on the dynamic environment model, inspection tasks with multiple flight constraints are allocated, and dynamic path planning is performed using the 3D scene model to achieve precise path planning. A secondary inspection is then performed based on the planned path to identify anomalies in the target area, achieving improved accuracy of inspection results and effective identification of anomalies. This solves the technical problem in related technologies where inspection methods based on historical data for path planning have poor adaptability and low accuracy.

[0101] The following is a detailed description with reference to another embodiment.

[0102] Example 2

[0103] The UAV-based dynamic inspection system provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in Embodiment 1 above. The specific implementation method and beneficial effects can be referred to the aforementioned method embodiment, and will not be repeated here.

[0104] Figure 3 This is a schematic diagram of an optional UAV-based dynamic inspection system according to an embodiment of the present invention, such as... Figure 3 As shown, the UAV-based dynamic inspection system may include: an acquisition unit 31, a modeling unit 32, an allocation unit 33, an optimization unit 34, and an inspection unit 35, wherein...

[0105] The acquisition unit 31 is used to acquire multi-source inspection data collected by the target UAV when it performs inspection operations on the target inspection area according to the preset inspection plan, and to perform multi-dimensional preprocessing on the multi-source inspection data to obtain spatiotemporal aligned data.

[0106] Modeling unit 32 is used to perform three-dimensional scene modeling of the target inspection area using spatiotemporal aligned data, and obtain a three-dimensional scene model and a dynamic environment model.

[0107] The allocation unit 33 is used to acquire inspection operation instructions and allocate inspection tasks with multiple flight constraints to the dynamic environment model based on the inspection operation instructions to obtain a task execution plan.

[0108] The optimization unit 34 is used to perform global path planning on the task execution plan based on the three-dimensional scene model to obtain the initial inspection path, and to perform local spatiotemporal collaborative path optimization on the initial inspection path to obtain the target inspection path of the target UAV.

[0109] The inspection unit 35 is used to remotely control the target UAV to perform optimized path inspection based on the target inspection path, obtain optimized path inspection data, perform anomaly detection on the optimized path inspection data, obtain anomaly detection results, and generate an inspection report of the target inspection area based on the anomaly detection results.

[0110] The aforementioned UAV-based dynamic inspection system acquires multi-source inspection data collected by the target UAV during its inspection of the target inspection area according to a preset inspection plan through acquisition unit 31, and performs multi-dimensional preprocessing on the multi-source inspection data to obtain spatiotemporally aligned data; modeling unit 32 uses the spatiotemporally aligned data to perform 3D scene modeling of the target inspection area to obtain a 3D scene model and a dynamic environment model; allocation unit 33 acquires inspection operation instructions and allocates inspection tasks with multiple flight constraints to the dynamic environment model based on the inspection operation instructions to obtain a task execution plan; optimization unit 34 performs global path planning on the task execution plan based on the 3D scene model to obtain an initial inspection path, and performs local spatiotemporal collaborative path optimization on the initial inspection path to obtain the target inspection path of the target UAV; inspection unit 35 remotely controls the target UAV to perform optimized path inspection based on the target inspection path to obtain optimized path inspection data, performs anomaly detection on the optimized path inspection data to obtain anomaly detection results, and generates an inspection report for the target inspection area based on the anomaly detection results.

[0111] This embodiment proposes a secondary inspection mechanism. A primary inspection is performed using a pre-planned route, and the primary inspection data is fully utilized for modeling, constructing a 3D scene model and a dynamic environment model. Based on the dynamic environment model, inspection tasks with multiple flight constraints are allocated, and dynamic path planning is performed using the 3D scene model to achieve precise path planning. A secondary inspection is then performed based on the planned path to identify anomalies in the target area, achieving improved accuracy of inspection results and effective identification of anomalies. This solves the technical problem in related technologies where inspection methods based on historical data for path planning have poor adaptability and low accuracy.

[0112] Furthermore, the modeling unit includes: a first segmentation module for segmenting the spatiotemporally aligned data to obtain point cloud sequences and multimodal image sequences; a first analysis module for analyzing the point cloud sequences and multimodal image sequences to obtain initial pose estimation, pixel-level semantic labels, multi-dimensional environmental information, and time-varying environmental factor fields; a first adjustment module for adjusting the reprojection error of the initial pose estimation to obtain accurate camera trajectories, and based on the accurate camera trajectories, performing registration fusion and triangulation on the point cloud sequences to obtain an initial 3D scene framework; a first modeling module for projecting pixel-level semantic labels onto the initial 3D scene framework to obtain inspection scene elements with category labels, and performing instance segmentation and parametric modeling on the inspection scene elements to obtain a 3D scene model; a first encoding module for performing multi-resolution octree encoding on the 3D scene model to obtain 3D scene storage data, and performing feature overlay on the multi-dimensional environmental information and 3D scene storage data to obtain an inspection environment representation; and a first integration module for integrating the spatiotemporal features of the inspection environment representation and the time-varying environmental factor field to obtain a dynamic environment model.

[0113] Furthermore, the first analysis module includes: a first matching submodule, used to extract point cloud features and perform inter-frame matching on the point cloud sequence to obtain an initial pose estimate; a first analysis submodule, used to perform semantic segmentation on the multimodal image sequence to obtain pixel-level semantic labels, and to perform time-series analysis on the multispectral data, temperature distribution map, and parameter distribution in the multimodal image sequence to obtain multidimensional environmental information; and a first extraction submodule, used to extract the time-series environmental parameter sequence from the multidimensional environmental information, and to perform interpolation and spatial mapping on the time-series environmental parameter sequence to obtain a continuous environmental parameter field, and to perform time-series analysis on the continuous environmental parameter field to obtain a time-varying environmental factor field.

[0114] Furthermore, the allocation unit includes: a first extraction module, used to extract the task features corresponding to the inspection operation from the dynamic environment model based on the inspection operation instructions, to obtain a multi-dimensional task feature vector; a first sorting module, used to perform cluster analysis and priority sorting on the multi-dimensional task feature vector to obtain the task execution order of the optimized path inspection; a first matching module, used to perform UAV performance parameter matching on the task execution order based on the UAV performance parameters of the target UAV, to obtain an initial task allocation scheme; and a first optimization module, used to perform multi-objective optimization and task decision analysis on the initial task allocation scheme to obtain a task execution scheme.

[0115] Furthermore, the optimization unit includes: a second analysis module, used to rasterize the 3D scene in the 3D scene model to obtain a multi-resolution probabilistic map of the target inspection area, and to perform topological analysis on the multi-resolution probabilistic map to obtain a critical path node network; a first search module, used to perform a heuristic search for initial inspection points on the critical path node network to obtain an initial global path, and to perform dynamic constraint checks and path optimization on the initial global path to obtain a smooth inspection path that satisfies the dynamic constraints; and a first evaluation module, used to perform time-varying wind field compensation on the smooth inspection path to obtain a wind field adaptive path for the target inspection area, and to perform energy consumption evaluation and multi-objective evaluation on the wind field adaptive path to obtain an initial inspection path.

[0116] Furthermore, the optimization unit also includes: a first decomposition module, used to perform spatiotemporal decomposition on the initial inspection path to obtain multiple local path segments and corresponding road segment time windows, and extract the environmental dynamic features within each road segment time window to obtain a time-varying environmental parameter set; a first prediction module, used to perform dynamic obstacle trajectory prediction on each local path segment based on the time-varying environmental parameter set to obtain a set of expected conflict points, and perform spatiotemporal density analysis on the set of expected conflict points to obtain high-risk areas and high-risk time periods, wherein a high-risk area refers to an area where the regional risk value is higher than a preset risk threshold, and a high-risk time period refers to a time period where the time period risk value is higher than a preset risk threshold; and a first planning module, used to perform multi-drone collaborative perception planning on the target UAV based on the high-risk areas and high-risk time periods to obtain the initial observation point distribution, and to perform initial... The initial observation point distribution is evaluated for information gain to obtain an optimized observation position sequence. The first fusion module is used to fuse the observation position sequence and local path segments to obtain an initial gain path. The initial gain path is then subjected to dynamic constraints and time window compatibility extension to obtain a set of candidate local paths that meet the time constraints. The first construction module is used to optimize the candidate local path set for multiple local parameters to obtain the optimal local path. Using the optimal local path, a dynamic artificial potential field that meets the preset time factors is constructed to obtain a time-varying obstacle avoidance force field. The second integration module is used to perform spatiotemporal collaborative optimization of the time-varying obstacle avoidance force field for multiple UAVs to obtain a collaborative obstacle avoidance strategy that meets the time windows of multiple UAVs and road segments. The optimal local path, collaborative obstacle avoidance strategy, and road segment time window are integrated to obtain the target inspection path of the target UAV.

[0117] Furthermore, the inspection unit includes: a first inspection module, used to perform remote global inspection of the target UAV based on the global inspection path in the target inspection path, to obtain global inspection data; a second inspection module, used to perform remote local inspection of the local inspection area corresponding to the local inspection path based on the local inspection path in the target inspection path, using the target UAV, to obtain local inspection data; and a second fusion module, used to perform spatiotemporal registration of the global inspection data and the local inspection data to obtain a multi-scale inspection dataset, and to perform multi-modal data fusion of the multi-scale inspection dataset to obtain optimized path inspection data with comprehensive feature representation.

[0118] It should be noted that the acquisition unit 31, modeling unit 32, allocation unit 33, optimization unit 34, and inspection unit 35 mentioned above correspond to steps S201 to S205 in Embodiment 1. The instances and application scenarios implemented by the above units and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules or units can also run as part of a device in the computer terminal 10 provided in Embodiment 1.

[0119] The invention will now be described in conjunction with another alternative embodiment.

[0120] Example 3

[0121] The present invention can also provide an electronic device. Figure 4 This is a hardware structure block diagram of an optional electronic device (or mobile device) for implementing a UAV-based dynamic inspection method according to an embodiment of the present invention, such as... Figure 4 As shown, the electronic device may include: one or more ( Figure 4 (Only one is shown) Processor 402, memory 404, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0122] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0123] The processor can access information and applications stored in the memory via a transmission device to execute the following steps: acquire multi-source inspection data collected by the target UAV during its inspection of the target inspection area according to a preset inspection plan, and perform multi-dimensional preprocessing on the multi-source inspection data to obtain spatiotemporally aligned data; use the spatiotemporally aligned data to perform 3D scene modeling of the target inspection area to obtain a 3D scene model and a dynamic environment model; acquire inspection operation instructions, and allocate inspection tasks with multiple flight constraints to the dynamic environment model based on the inspection operation instructions to obtain a task execution plan; based on the 3D scene model, perform global path planning on the task execution plan to obtain an initial inspection path, and perform local spatiotemporal collaborative path optimization on the initial inspection path to obtain the target inspection path of the target UAV; based on the target inspection path, remotely control the target UAV to perform optimized path inspection, obtain optimized path inspection data, perform anomaly detection on the optimized path inspection data to obtain anomaly detection results, and generate an inspection report for the target inspection area based on the anomaly detection results.

[0124] This invention provides a dynamic inspection scheme. It performs a primary inspection along a pre-planned route, fully utilizing the data from this primary inspection to build a 3D scene model and a dynamic environment model. Based on the dynamic environment model, it establishes inspection task allocation with multiple flight constraints and performs dynamic path planning using the 3D scene model, achieving precise path planning. A secondary inspection is then performed based on the planned path to identify anomalies in the target area. This improves the accuracy of inspection results and effectively identifies anomalies. Furthermore, it solves the technical problem in related technologies where inspection methods based on historical data for path planning have poor adaptability and low accuracy.

[0125] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 4 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.

[0126] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0127] The invention will now be described in conjunction with another alternative embodiment.

[0128] Example 4

[0129] This invention also provides a computer-readable storage medium. Optionally, in this invention, the computer-readable storage medium can be used to store the program code executed by the UAV-based dynamic inspection method provided in Embodiment 1.

[0130] Optionally, in this embodiment of the invention, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0131] This invention also provides a computer program product, which, when executed on a data processing device, is suitable for executing the steps of a dynamic inspection method based on a drone: acquiring multi-source inspection data collected by the target drone during inspection of a target inspection area according to a preset inspection plan, and performing multi-dimensional preprocessing on the multi-source inspection data to obtain spatiotemporally aligned data; using the spatiotemporally aligned data to perform three-dimensional scene modeling of the target inspection area to obtain a three-dimensional scene model and a dynamic environment model; acquiring inspection operation instructions, and allocating inspection tasks with multiple flight constraints to the dynamic environment model based on the inspection operation instructions to obtain a task execution plan; based on the three-dimensional scene model, performing global path planning on the task execution plan to obtain an initial inspection path, and performing local spatiotemporal collaborative path optimization on the initial inspection path to obtain the target inspection path of the target drone; based on the target inspection path, remotely controlling the target drone to perform optimized path inspection to obtain optimized path inspection data, and performing anomaly detection on the optimized path inspection data to obtain anomaly detection results, and generating an inspection report for the target inspection area based on the anomaly detection results.

[0132] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0133] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0135] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0136] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0137] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0138] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A dynamic inspection method based on unmanned aerial vehicles (UAVs), characterized in that, include: The system acquires multi-source inspection data collected by the target UAV when it performs inspection operations on the target inspection area according to a preset inspection plan, and performs multi-dimensional preprocessing on the multi-source inspection data to obtain spatiotemporally aligned data. The spatiotemporal alignment data is used to perform three-dimensional scene modeling of the target inspection area, resulting in a three-dimensional scene model and a dynamic environment model. Obtain inspection operation instructions, and based on the inspection operation instructions, allocate inspection tasks with multiple flight constraints to the dynamic environment model to obtain a task execution plan; Based on the three-dimensional scene model, global path planning is performed on the task execution plan to obtain an initial inspection path, and local spatiotemporal collaborative path optimization is performed on the initial inspection path to obtain the target inspection path of the target UAV. Based on the target inspection path, the target UAV is remotely controlled to perform optimized path inspection, obtain optimized path inspection data, and perform anomaly detection on the optimized path inspection data to obtain anomaly detection results. Based on the anomaly detection results, an inspection report of the target inspection area is generated.

2. The method according to claim 1, characterized in that, The steps for performing 3D scene modeling on the target inspection area using the spatiotemporal aligned data to obtain a 3D scene model and a dynamic environment model include: The spatiotemporally aligned data is segmented to obtain point cloud sequences and multimodal image sequences; The point cloud sequence and multimodal image sequence are analyzed to obtain initial pose estimation, pixel-level semantic labels, multi-dimensional environmental information, and time-varying environmental factor field. The reprojection error of the initial pose estimation is adjusted to obtain the accurate camera trajectory. Based on the accurate camera trajectory, the point cloud sequence is registered, fused, and triangulated to obtain the initial 3D scene framework. The pixel-level semantic tags are projected onto the initial 3D scene framework to obtain inspection scene elements with category tags. The inspection scene elements are then segmented into instances and parametrically modeled to obtain the 3D scene model. The three-dimensional scene model is encoded using a multi-resolution octree to obtain three-dimensional scene storage data. The multi-dimensional environmental information and the three-dimensional scene storage data are then superimposed to obtain an inspection environment representation. The dynamic environment model is obtained by integrating the spatiotemporal features of the inspection environment representation and the time-varying environmental factor field.

3. The method according to claim 2, characterized in that, The steps for analyzing the point cloud sequence and multimodal image sequence to obtain initial pose estimation, pixel-level semantic labels, multi-dimensional environmental information, and time-varying environmental factor field include: Point cloud features are extracted and inter-frame matching is performed on the point cloud sequence to obtain an initial pose estimate; Semantic segmentation is performed on the multimodal image sequence to obtain pixel-level semantic labels, and time-series analysis is performed on the multispectral data, temperature distribution map, and parameter distribution in the multimodal image sequence to obtain multidimensional environmental information; The time-series environmental parameter sequence is extracted from the multi-dimensional environmental information, and the time-series environmental parameter sequence is interpolated and spatially mapped to obtain a continuous environmental parameter field. The continuous environmental parameter field is then subjected to time series analysis to obtain a time-varying environmental factor field.

4. The method according to claim 1, characterized in that, The steps for allocating inspection tasks with multiple flight constraints to the dynamic environment model based on the inspection operation instructions, and obtaining the task execution plan, include: Based on the inspection operation instructions, the task features corresponding to the inspection operation are extracted from the dynamic environment model to obtain a multi-dimensional task feature vector. Cluster analysis and task priority ranking are performed on the multidimensional task feature vectors to obtain the task execution order of optimized path inspection; Based on the performance parameters of the target UAV, the UAV performance parameters are matched to the task execution order to obtain an initial task allocation scheme. The initial task allocation scheme is optimized through multi-objective optimization and task decision analysis to obtain the task execution scheme.

5. The method according to claim 1, characterized in that, Based on the 3D scene model, the steps for global path planning of the task execution scheme to obtain the initial inspection path include: The three-dimensional scene in the three-dimensional scene model is rasterized to obtain a multi-resolution probabilistic map of the target inspection area, and a topology analysis is performed on the multi-resolution probabilistic map to obtain the critical path node network. A heuristic search for initial inspection points is performed on the critical path node network to obtain an initial global path. Then, dynamic constraint checks and path optimization are performed on the initial global path to obtain a smooth inspection path that satisfies the dynamic constraints. Time-varying wind field compensation is performed on the smoothed inspection path to obtain the wind field adaptive path of the target inspection area. Energy consumption assessment and multi-objective evaluation are then performed on the wind field adaptive path to obtain the initial inspection path.

6. The method according to claim 5, characterized in that, The steps of performing local spatiotemporal cooperative path optimization on the initial inspection path to obtain the target inspection path of the target UAV include: The initial inspection path is decomposed in time and space to obtain multiple local path segments and corresponding road segment time windows. The environmental dynamic features within each road segment time window are extracted to obtain a time-varying environmental parameter set. Based on the time-varying environmental parameter set, dynamic obstacle trajectory prediction is performed on each local path segment to obtain a set of expected conflict points. Spatiotemporal density analysis is then performed on the set of expected conflict points to obtain high-risk areas and high-risk time periods. The high-risk area refers to an area where the regional risk value is higher than a preset risk threshold, and the high-risk time period refers to a time period where the time period risk value is higher than the preset risk threshold. Based on the high-risk area and the high-risk time period, multi-drone collaborative perception planning is performed on the target UAV to obtain the initial observation point distribution, and the information gain of the initial observation point distribution is evaluated to obtain the optimized observation position sequence. The observed location sequence and the local path segment are fused to obtain an initial gain path. The initial gain path is then subjected to dynamic constraints and time window compatibility extension to obtain a set of alternative local paths that meet the time constraints. The candidate local path set is optimized by multiple local parameters to obtain the optimal local path. The optimal local path is then used to construct a dynamic artificial potential field that satisfies a preset time factor, thus obtaining a time-varying obstacle avoidance force field. The time-varying obstacle avoidance force field is optimized by multi-UAV spatiotemporal collaboration to obtain a collaborative obstacle avoidance strategy that satisfies the time windows of multiple UAVs and road segments. The optimal local path, the collaborative obstacle avoidance strategy and the road segment time window are integrated to obtain the target inspection path of the target UAV.

7. The method according to claim 1, characterized in that, Based on the target inspection path, the steps of remotely controlling the target UAV to perform optimized path inspection and obtain optimized path inspection data include: Based on the global inspection path in the target inspection path, the target UAV is remotely inspected globally to obtain global inspection data. Based on the local inspection path in the target inspection path, the target UAV is used to perform remote local inspection of the local inspection area corresponding to the local inspection path to obtain local inspection data. Spatiotemporal registration is performed on the global inspection data and the local inspection data to obtain a multi-scale inspection dataset. Multimodal data fusion is then performed on the multi-scale inspection dataset to obtain optimized path inspection data with comprehensive feature representation.

8. A dynamic inspection system based on unmanned aerial vehicles (UAVs), characterized in that, include: The acquisition unit is used to acquire multi-source inspection data collected by the target UAV when it performs inspection operations on the target inspection area according to the preset inspection plan, and to perform multi-dimensional preprocessing on the multi-source inspection data to obtain spatiotemporal aligned data. The modeling unit is used to perform three-dimensional scene modeling on the target inspection area using the spatiotemporal alignment data, and obtain a three-dimensional scene model and a dynamic environment model. The allocation unit is used to acquire inspection operation instructions and allocate inspection tasks with multiple flight constraints to the dynamic environment model based on the inspection operation instructions to obtain a task execution plan. The optimization unit is used to perform global path planning on the task execution plan based on the three-dimensional scene model to obtain an initial inspection path, and to perform local spatiotemporal collaborative path optimization on the initial inspection path to obtain the target inspection path of the target UAV. The inspection unit is used to remotely control the target UAV to perform optimized path inspection based on the target inspection path, obtain optimized path inspection data, perform anomaly detection on the optimized path inspection data, obtain anomaly detection results, and generate an inspection report for the target inspection area based on the anomaly detection results.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the dynamic inspection method based on any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the UAV-based dynamic inspection method according to any one of claims 1 to 7.

Images