Unmanned aerial vehicle route planning method and device for wharf inspection

Abstract

The invention discloses an unmanned aerial vehicle route planning method for wharf inspection. The method comprises the following steps: acquiring reference map layer data through hierarchical fusion and verification based on multi-source geographic data; aggregating multi-source data into a data pool by constructing an index and managing a life cycle; analyzing result data in real time through computer vision and streaming processing based on the data pool, wherein the analysis result data at least comprises abnormal events and alarms; based on the analysis result data, generating and obtaining inspection data through a preset template and a natural language, wherein the inspection data comprises an inspection report and an intelligent abstract; acquiring route planning data through kernel density estimation and adaptive sampling based on the event distribution hot spots and violation frequency data of the inspection data; acquiring task data of an air route task based on the air route task of the air route planning data, updating the air route planning data and updating the data pool; and updating the inspection strategy based on the data pool through iteration of the model and the rule.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) application and autonomous route planning technology, and particularly relates to a UAV route planning method and device for port inspection. Background Technology

[0002] In industrial settings such as docks and ports, drones, with their advantages of maneuverability, flexibility, and high operational efficiency, have been gradually applied to core operational processes such as inspection, monitoring, and surveying, becoming an important technological means to improve port operation and management. However, as a core prerequisite for drone operations, flight path planning technology still has significant limitations. These limitations directly restrict the improvement of drone operational efficiency, the guarantee of operational safety, and the large-scale promotion and application in dock scenarios, becoming a key technological bottleneck for industry development. Currently, drone flight path planning technologies applied in dock scenarios can be mainly divided into the following three categories, and each type of technology has unavoidable defects: Firstly, there is the manual planning method. This is the traditional basic planning approach, relying on operators' personal experience to manually add waypoints one by one in the map interface of drone ground station software such as DJIPilot and DJIGSPro, and manually set key operational parameters such as flight altitude, flight speed, and shooting actions to ultimately form a complete flight path. This method is entirely dependent on manual operation, which is not only inefficient in planning, but also highly dependent on the operator's experience for the rationality of the flight path. It is prone to safety hazards due to human judgment errors and cannot meet the needs of large-scale port operations.

[0003] Secondly, there is the semi-automatic tool-assisted planning method. This method utilizes the surround and polygon scan functions of DJIGS Pro or the basic automation functions provided by third-party planning software such as Pix4Dcapture to assist in completing part of the route planning process. Compared with purely manual planning, although it reduces the intensity of manual operation to a certain extent, it still requires human intervention in setting core parameters and judging planning logic. The degree of automation is limited, and it cannot achieve fully autonomous route planning.

[0004] Thirdly, there is the automated planning method based on a pre-built rule base. This method is a relatively advanced planning technology. It requires the pre-construction of digital models such as two-dimensional maps or three-dimensional point cloud models for the terminal. Information such as fixed no-fly zones and obstacle locations are manually marked on the model to form a pre-built rule base. The system then automatically generates flight routes based on this rule base and geographic information. While this method improves automation, it has significant limitations: Firstly, environmental elements in a terminal setting, such as container stacking and ship berthing, are highly dynamic, making it difficult to update the pre-built rule base in real time to match environmental changes, resulting in insufficient flight route adaptability. Secondly, the construction of digital models and the maintenance of the rule base require continuous and substantial manpower investment, and the development of models and rule bases needs to be repeated for different terminal scenarios, resulting in extremely poor flexibility.

[0005] In summary, existing drone route planning technologies suffer from insufficient efficiency and safety due to over-reliance on manual labor, limited automation to handle dynamic scenarios, or high maintenance costs and poor flexibility, making them unsuitable for port operations. Therefore, the industry urgently needs a technical solution that deeply integrates with the characteristics of port scenarios and can autonomously generate safe, efficient, and precise routes based on operational objectives. This invention is proposed to address the aforementioned shortcomings of existing technologies. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a method and apparatus for UAV route planning in port inspection. It achieves accurate construction of baseline map layer data through hierarchical fusion and verification of multi-source geographic data; effectively integrates data by aggregating multi-source data into a data pool and managing indexes; promptly acquires abnormal events and alarms through real-time analysis of results using computer vision and streaming processing; efficiently generates inspection reports and intelligent summaries through preset templates and natural language generation; scientifically formulates route planning data by combining kernel density estimation and adaptive sampling with event distribution hotspots and violation frequency data; dynamically optimizes route planning data and the data pool through route task data collection and updating; and continuously improves inspection strategies through iterative updates of models and rules.

[0007] The first aspect of the present invention provides a method for planning the flight path of an unmanned aerial vehicle (UAV) for dock inspection, comprising: Baseline map layer data is obtained by hierarchical fusion and verification based on multi-source geographic data; By building indexes and managing lifecycles, multi-source data can be aggregated into a data pool; Based on the data pool, the results are analyzed in real time using computer vision and streaming processing. The results include at least abnormal events and alarms. Based on the analysis results, inspection data is generated using preset templates and natural language, and the inspection data includes inspection reports and intelligent summaries. Based on the event distribution hotspots and violation frequency data of the inspection data, route planning data is obtained through kernel density estimation and adaptive sampling. Based on the route planning data, the route task collects the task data of the route task and updates the route planning data and the data pool; The inspection strategy is updated iteratively based on the data pool and the model and rules.

[0008] Preferably, the step of obtaining baseline map layer data based on multi-source geographic data through hierarchical fusion and verification further includes: Obtain map data of the dock, wherein the zoom level of the map data is larger than that of a tile, and the map data includes at least the road network and the outline of the fixed buildings of the dock; Based on the map data of the wharf, the image data collected by the UAV, and the ground control coordinate data, the baseline map layer data is obtained by extracting features and aligning features at a preset scale through the hierarchical feature pyramid network of the encoder. The initial baseline map layer data integrates geometrically consistent feature information from multiple sources. Based on the map data of the terminal and the initial reference map layer data, the optimal geometric correction parameters are obtained by iterative optimization through an adaptive transformation model with added structural constraints, and the reference map layer data is updated. The structural constraints include at least the grid structure constraints of the container yard, the linear geometric constraints of the quay crane track, and the topological connectivity constraints of the road network. A time-series dataset is constructed based on the map data of the wharf and the image data collected by UAVs. A dynamic loss function is constructed through a registration algorithm guided by change detection to adjust the network hyperparameters and update the baseline map layer data. The updated base map layer data is obtained by weighting different feature data based on different versions of the base map layer data using a probabilistic fusion algorithm.

[0009] Preferably, the calculation expression for obtaining the updated baseline map layer data by weighting and updating different feature data through a probabilistic fusion algorithm is as follows: In the formula, The location data with historical version index i is (x, y) weighted data, N is the total number of versions of the base map layer data involved in the calculation, and T is the time series length. These are the uncertainty estimates for historical version index i and position data (x, y); The calculation expression for the baseline map layer data is: In the formula, The feature values ​​are obtained through probability fusion from the location data (x, y). The feature value of the position data (x, y) with index i in the historical version.

[0010] Preferably, the step of analyzing the results data in real time using computer vision and streaming processing based on the data pool further includes: A real-time data stream is constructed using video stream data transmitted in real time from drones and monitoring data from fixed cameras at the dock. Based on the real-time data stream, the hyperparameters of the real-time inference model are updated by constructing a loss function for multi-target nodes of edge computing nodes and backpropagating the gradient of the loss function. The loss function includes target detection loss, semantic segmentation loss, and multi-target tracking loss.

[0011] Preferably, the step of obtaining route planning data based on the event distribution hotspots and violation frequency data of the inspection data through kernel density estimation and adaptive sampling further includes: Based on event location data and data from historical inspection data, a historical inspection distribution map and violation distribution data are constructed. Based on the historical inspection distribution map and the violation distribution data, a continuous probability density distribution map is generated using a multidimensional kernel density estimation algorithm. The set of cruise points is obtained based on the probability density distribution map using an importance sampling algorithm; Based on the cruise point set and the initial sampling results, the sampling accuracy is increased and the sampling results are updated using an adaptive grid subdivision algorithm; Based on the sampling results, the optimal route planning data is obtained through a constrained path optimization algorithm; The route planning data is updated by satisfying the constraints based on the constraints of the dock environment and the constraints of the UAV.

[0012] The preferred expression for multidimensional kernel density estimation is: In the formula, Let N be the probability density of the abnormal event at coordinates (x, y), and let N be the total number of versions of the historical inspection data used in the calculation. The different geometric features of the terminal include at least quay cranes, rails, and container yards. This is the weighted data for the violation event with version index number i, based on historical inspection data. For the feature dataset based on coordinates (x, y), For different types of geometric features of the dock, the smoothed values ​​are given in the most stable direction. This refers to the geodetic distance during inspections at the dock.

[0013] Preferably, the step of obtaining the cruise point set based on the probability density distribution map using an importance sampling algorithm further includes: Initial sampling probability distribution data is obtained based on the probability density distribution map, and the sampling probability distribution data is used as the basis for decision-making in UAV inspection missions. The initial sampling probability is successively refined through a multi-level threshold grid to obtain a multi-scale sampling grid, and a constraint-aware sampling algorithm is used to obtain a set of candidate sampling points. The optimal set of cruise points is obtained by screening candidate sampling point sets through allocation strategies, calibrating the coverage of critical safety areas at the dock, and performing multi-objective optimization.

[0014] Preferably, the step of collecting task data for the route planning data and updating the data pool based on the route planning data further includes: Based on the aforementioned flight path mission, the UAV performs inspection tasks and collects multimodal mission data in real time. The multimodal mission data includes at least image data, location data, and environmental data. The multimodal task data is filtered and updated based on geometric quality assessment, temporal quality assessment, and semantic quality assessment. The corresponding inspection quality score is obtained based on the multimodal task data, and the data pool is updated based on the multimodal task data.

[0015] Preferably, the step of iteratively updating the inspection strategy based on the data pool through models and rules further includes: Historical and current inspection data from the data pool are used to quantify the quality score of the current inspection strategy through an indicator system. Based on the policy evaluation results, the computer vision model and optimization algorithm are continuously optimized through online learning algorithms.

[0016] A second aspect of the present invention provides a drone route planning device for dock inspection, comprising: The map building module is used to obtain baseline map layer data based on multi-source geographic data through layered fusion and verification; The data management module is used to aggregate multi-source data into a data pool by building indexes and managing the lifecycle; The real-time analysis module is used to analyze the results data in real time based on the data pool using computer vision and streaming processing. The report generation module is used to generate inspection data based on the analysis results data using preset templates and natural language. The route planning module is used to obtain route planning data based on the event distribution hotspots and violation frequency data of the inspection data through kernel density estimation and adaptive sampling. The task update module is used to collect task data of the route task based on the route planning data, update the route planning data, and update the data pool; The strategy optimization module is used to update the inspection strategy based on the data pool through iterative updates of models and rules.

[0017] This invention, by employing the above technical solutions, possesses the following advantages and positive effects compared to existing technologies: It achieves accurate construction of baseline map layer data through hierarchical fusion and verification of multi-source geographic data; it achieves effective data integration through the aggregation of multi-source data into a data pool and index management; it enables timely acquisition of abnormal events and alarms through real-time analysis of results data using computer vision and streaming processing; it achieves efficient generation of inspection reports and intelligent summaries through preset templates and natural language generation; it enables the scientific formulation of route planning data through kernel density estimation and adaptive sampling combined with event distribution hotspots and violation frequency data; it achieves dynamic optimization of route planning data and the data pool through route task data collection and updating; and it enables continuous improvement of inspection strategies through iterative updates of models and rules. Attached Figure Description

[0018] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is the main flowchart of a drone route planning method for dock inspection according to the present invention. Detailed Implementation

[0019] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise ratios, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0020] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0021] First Embodiment The first aspect of the present invention provides a method for planning the flight path of an unmanned aerial vehicle (UAV) for dock inspection, comprising: Baseline map layer data is obtained by hierarchical fusion and verification based on multi-source geographic data; By building indexes and managing lifecycles, multi-source data can be aggregated into a data pool; Based on the data pool, the results data are analyzed in real time using computer vision and streaming processing. The analysis results data include at least abnormal events and alarms. Based on the analysis results, inspection data is generated using preset templates and natural language. The inspection data includes inspection reports and intelligent summaries. Based on the event distribution hotspots and violation frequency data from inspection data, route planning data is obtained through kernel density estimation and adaptive sampling. Based on route planning data, route tasks collect task data for route tasks and update route planning data and data pool; The inspection strategy is updated iteratively through models and rules based on the data pool.

[0022] Through hierarchical fusion and verification of multi-source geographic data, high-precision construction of the baseline map layer data was achieved. By constructing an index and a lifecycle management mechanism, multi-source data was effectively integrated and formed into a unified data pool. Real-time data analysis using computer vision and streaming technology enabled rapid identification of abnormal events and the issuance of alarms. Based on preset templates and natural language generation technology, structured inspection data containing inspection reports and intelligent summaries was automatically generated. Combining event distribution hotspots and violation frequencies, kernel density estimation and adaptive sampling algorithms were used to optimize route planning, improving inspection efficiency and coverage accuracy. By executing route tasks to collect task data, the route planning and data pool were dynamically updated, achieving closed-loop feedback within the system. Finally, through an iterative learning mechanism of models and rules, the inspection strategy was continuously optimized, enhancing the system's adaptability and intelligence. Lifecycle management, based on the baseline map layer data, was used to formulate corresponding lifecycle strategies based on the rate of value decay. The index included time indexes, spatial indexes, and device indexes to construct the first data set. The time index partitioned and sorted data by collection time for rapid retrieval by time range. The spatial index, through geospatial dimension indexing, associated data with location data for rapid retrieval of data features corresponding to the relevant area. Equipment Source Index: Categorized by drone ID, camera ID, etc. Hot data includes newly collected, currently being processed, and recent high-value event data, stored in high-speed storage. Warm data consists of historical task data that requires occasional access, stored in lower-performance storage such as HDD arrays. Cold data is long-term archived or compliance-compliant data, which can be stored in object storage or tape libraries. Computer vision algorithms are used to automatically analyze images and video streams in the data pool to identify violations, dynamic obstacles, and other critical events. Violations include at least personnel intrusion, equipment malfunctions, and non-compliant cargo stacking. The computer vision algorithms include at least object detection and semantic segmentation. Inspection data includes at least the type of violation, its location, time, frequency statistics, and on-site image evidence. Inspection reports, through automated technology, transform structured and unstructured analytical data into comprehensive inspection reports and intelligent summaries that are formatted correctly, accurate, highlight key points, and include visual evidence. This significantly reduces the workload of manual report writing and supports rapid decision-making. The inspection report is generated by designing a report template. A pre-designed template in Word or HTML can be used, including titles, chapters, paragraphs, tables, and image placeholders. The template uses placeholder variables such as {{report_date}}, {{total_events}}, and {{event_table}} to bind pre-processed data to these template variables. A template engine (such as Apache FreeMarker for Java or Jinja2 for Python) then populates the data into the template, generating the final Word, PDF, or HTML report.The inspection report uses natural language generation (NLG) technology, pre-defining sentence templates for different types of analysis results. For example, input could be: (Event Type = Illegal Parking, Location = Zone B2, Time = 2023-10-27 14:30:00), rule: At [time], an [event type] was found at [location]. Output: At 2:30 PM on October 27, 2023, an illegal parking incident was found in Zone B2. Deep learning-based NLG: Model selection: Fine-tuning using pre-trained text generation models (such as GPT series, T5, etc.). Training data: Paired data with a large amount of structured text descriptions is required. A basic corpus can be generated first using rule-based NLG, and then manually refined to form a high-quality training set. Input / output: Input structured data (such as JSON format) into the model, and the model directly generates fluent natural language paragraphs. The inspection report supports multiple output formats, such as PDF (for printing and archiving), HTML (for web viewing), and Word (for secondary editing). Scheduled triggering: Periodic reports can be automatically generated daily / weekly / monthly. Event-driven: When a major anomaly occurs, a special bulletin is immediately generated and sent. Intelligent distribution: Based on the importance of the report content, it is automatically pushed to relevant responsible persons via email, messaging robots (such as WeChat Work, DingTalk), etc.

[0023] Preferably, the step of obtaining baseline map layer data based on multi-source geographic data through hierarchical fusion and verification further includes: Obtain map data of the dock, with the map data zoomed in size larger than the tile, and the map data should at least include the dock's road network and the outline of fixed buildings; Based on the map data of the dock, the image data collected by the UAV, and the ground control coordinate data, the baseline map layer data is obtained by extracting features and aligning features at a preset scale through the hierarchical feature pyramid network of the encoder. The initial baseline map layer data is fused with geometrically consistent feature information from multiple sources. Based on the terminal's map data and the initial baseline map layer data, an adaptive transformation model is used to incorporate structural constraints for iterative optimization to obtain the optimal geometric correction parameters and update the baseline map layer data. The structural constraints include at least the grid structure constraints of the container yard, the linear geometric constraints of the quay crane tracks, and the topological connectivity constraints of the road network. A time-series dataset was constructed based on map data of the dock and image data collected by drones. A dynamic loss function was constructed using a registration algorithm guided by change detection to adjust the network hyperparameters and update the baseline map layer data. Updated baseline map layer data is obtained by weighting and updating different feature data based on different versions of baseline map layer data using a probabilistic fusion algorithm.

[0024] By acquiring high-resolution terminal map data and fusing it with multi-source information, accurate modeling of key geographic elements such as road networks and building outlines is achieved. Multi-scale feature extraction and alignment are performed using a hierarchical feature pyramid network of the encoder, effectively integrating UAV imagery and ground control coordinate data to generate a geometrically consistent initial baseline map layer. An adaptive transformation model is introduced, combined with structural constraints such as container yard grids, quay crane track linear structures, and road topological connectivity, to iteratively optimize geometric correction parameters, significantly improving the spatial accuracy and consistency of the map. A time-series dataset and a change detection-guided registration algorithm are used to dynamically adjust network hyperparameters, enhancing the map's adaptability to environmental changes. Finally, a probabilistic fusion algorithm is used to weighted update different versions of map data, achieving continuous evolution and highly reliable output of the baseline map layer. In this embodiment, the multi-source geographic data can utilize open-source low-precision satellite tile map services such as Tianditu, with a resolution of 1-5 meters per pixel. The scope of multi-source geographic data includes: road networks, fixed building outlines, and invariant features. The road network includes main roads, auxiliary roads, and lane markings within the wharf. Fixed building outlines include warehouses, office buildings, wharf cranes, and boundaries of fixed container yards. Invariant features include large parking lot outlines, fixed green belts, and shorelines. The multi-source geographic data is in PNG / JPEG format and includes location data. An ensemble correction method eliminates distortions caused by satellite sensor attitude and terrain undulations, ensuring the geometric accuracy of the map. Orthorectification can be performed using ground control points or based on known, accurate digital surface models. The baseline map layer data unifies the multi-source geographic data into a single global or local coordinate system. The coordinate system can adopt WGS84UTM to achieve a unified coordinate system, providing a standard data source for subsequent spatial calculations. The multi-source geographic data here includes low-altitude reconnaissance imagery data. Specifically, in the initial stage of the project, a drone can be dispatched for a low-altitude, non-missionary, rapid flight to take a small number of photos of key areas. A high-precision orthophoto map (DOM) and digital surface model (DSM) are generated using the Structure for Motion Restoration (SfM) algorithm. This high-precision DOM can serve as a gold standard to verify and correct the accuracy of vector features extracted from low-precision satellite imagery. Low-altitude reconnaissance imagery data is then compared with the extracted vector features to the high-precision DOM or manually labeled real data to calculate indicators such as positional accuracy and completeness, ensuring the reliability of the base map.

[0025] Preferably, the calculation expression for obtaining the updated baseline map layer data by weighting and updating different feature data through a probabilistic fusion algorithm is as follows: In the formula, The location data for historical version index i is (x, y) weighted data, N is the total number of versions of the baseline map layer data involved in the calculation, and T is the time series length. These are the uncertainty estimates for historical version index i and position data (x, y); The calculation expression for the baseline map layer data is: In the formula, The feature values ​​are obtained through probability fusion from the location data (x, y). The feature value of the position data (x, y) with index i in the historical version.

[0026] By using a probabilistic fusion algorithm to weight and update different feature data, dynamic integration and uncertainty quantification of historical versions of the baseline map layer data are achieved, effectively improving the consistency of feature representation of location data (x, y) under multi-source heterogeneous information. By utilizing weighted data and uncertainty estimation mechanisms, the credibility contribution of each historical version is rationally allocated, reducing the impact of low-quality or outdated data and enhancing the robustness and spatiotemporal continuity of the baseline map layer. The introduction of a time series length T supports the combination of long-term evolution analysis and short-term change response, making map updates both stable and flexible. The final updated baseline map layer data possesses higher accuracy, reliability, and adaptability, providing accurate and stable geospatial foundation support for subsequent UAV inspection tasks. The baseline map layer data is stored in a hierarchical structure. The first layer is a raster layer, including a satellite base map used for visualization background; the satellite base map is the original low-precision satellite imagery. The second layer is a vector line layer, including the road network, further including road centerlines and lane information. The third layer is a vector polygon layer, including building outlines, which include polygons of all fixed buildings. The fourth layer is a vector surface layer, which includes no-fly zones / restricted flight zones and further describes the areas generated based on building outlines and safety management rules. The fifth layer is a vector point layer, which includes navigation feature points and alternative landmarks used for UAV visual navigation.

[0027] Preferably, the step of analyzing the results data in real time based on the data pool using computer vision and streaming processing further includes: A real-time data stream is constructed using video stream data transmitted in real time from drones and monitoring data from fixed cameras at the dock. Based on real-time data streams, the hyperparameters of the real-time inference model are updated by constructing a loss function for multi-target nodes on edge computing nodes and backpropagating the gradient of the loss function. The loss function includes target detection loss, semantic segmentation loss, and multi-target tracking loss.

[0028] By integrating real-time video streams from drones with monitoring data from fixed cameras at the dock, a high-efficiency streaming data ingestion engine is constructed to achieve seamless access and real-time synchronization of multi-source sensing information. A lightweight inference framework is deployed based on edge computing nodes, and a jointly optimized multi-task loss function is constructed by combining target detection loss, semantic segmentation loss, and multi-target tracking loss to improve the recognition accuracy and tracking stability of key targets such as ships, vehicles, and personnel in complex scenarios. The gradient backpropagation mechanism is used to dynamically adjust the model's hyperparameters, enabling the real-time inference model to have online learning capabilities and continuously adapt to environmental changes and emerging behavioral patterns. Ultimately, this significantly enhances the ability to detect abnormal events (such as unauthorized operations and personnel intrusion) and the speed of alarm response, providing highly timely and reliable visual analysis support for intelligent dock inspection. This embodiment uses various front-end devices, such as drone-borne sensors and fixed dock monitoring cameras, to collect image, video, and abnormal event data in real time, and transmits them to a central data pool for centralized storage and management. The collected image, video, and abnormal event data includes unstructured data, semi-structured data, and structured data. Unstructured data includes JPEG, PNG, and RAW image data, and MP4 and AVI video streams encoded with H.264 and H.265. Semi-structured data includes metadata collected by sensors, stored in JSON, XML, or custom binary formats. Metadata includes at least timestamps, GPS coordinates, UAV attitude, and sensor parameters, with UAV attitude including at least pitch, roll, and yaw. Structured data includes anomalous event messages, which include at least event type, location, time, and device ID. For example, a timelist structure for structured data could be: [Event Type, Location (X, Y), Timestamp, Confidence, Associated Image / Video ID], with statistical results like: {"Illegal Parking": 15, "Intrusion": 3,...}. Semi-structured / unstructured data can be evidence images, video clips, or summaries or keyframes extracted from collected images, videos, and anomalous event data.

[0029] Preferably, the step of obtaining route planning data based on the event distribution hotspots and violation frequency data of the inspection data through kernel density estimation and adaptive sampling further includes: Based on the event location data and data from historical inspection data, a historical inspection distribution map and violation distribution data are constructed. Based on the historical inspection distribution map and violation distribution data, a continuous probability density distribution map is generated through a multidimensional kernel density estimation algorithm. The cruise point set is obtained based on the probability density distribution map using an importance sampling algorithm; Based on the cruise point set and initial sampling results, an adaptive grid subdivision algorithm is used to increase sampling accuracy and update the sampling results; The optimal route planning data is obtained through a constrained path optimization algorithm based on the sampling results; Based on the constraints of the dock environment and the constraints of the UAV, the route planning data is updated by satisfying the constraints.

[0030] By integrating event location and violation frequency information from historical inspection data, a high-precision historical inspection distribution map and violation distribution dataset are constructed. A multidimensional kernel density estimation algorithm is used to generate a continuous probability density distribution map, accurately identifying high-risk areas and hotspot event clusters. Based on the probability density distribution map, an importance sampling algorithm is employed to efficiently select key patrol point sets, ensuring that the UAV flight path covers the most potentially risky areas, improving the targeting and resource utilization of inspections. An adaptive grid subdivision algorithm dynamically refines the sampling granularity, significantly improving spatial resolution and positioning accuracy while maintaining computational efficiency. Furthermore, a constraint path optimization algorithm is introduced, comprehensively considering the port environment (such as obstacles and no-fly zones) and UAV performance limitations (such as endurance and communication range) to generate a safe, feasible, and optimal route planning scheme. Finally, a constraint satisfaction mechanism is used to correct the route data in real time, achieving dynamic adjustment and robustness enhancement of the route, comprehensively improving the intelligence level and task execution efficiency of port inspections. Based on event distribution hotspots and violation frequency data in the inspection data, optimized patrol points and inspection plans are automatically generated. The system can dynamically construct inspection routes based on a pre-set rule base (such as inspection priority and safety distance constraints). The static, fixed inspection routes will be upgraded to a dynamic, data-driven, and adaptive intelligent mission planning system. The core of this system is to shift from "time-based patrols" to "on-demand patrols," maximizing the efficiency and effectiveness of limited flight resources.

[0031] The preferred expression for multidimensional kernel density estimation is: In the formula, Let be the probability density of the abnormal event at coordinates (x, y), and N be the total number of versions of historical inspection data involved in the calculation. The different geometric features of the terminal include at least quay cranes, rails, and container yards. This is the weighted data for the violation event with version index number i, based on historical inspection data. For the feature dataset based on coordinates (x, y), For different types of geometric features of the dock, the smoothed values ​​are given in the most stable direction. This refers to the geodetic distance during inspections at the dock.

[0032] By introducing a multidimensional kernel density estimation algorithm, this method comprehensively considers the spatial distribution of abnormal events, the weight of historical violations, and the spatial influence of key geometric features of the wharf, achieving continuous and refined modeling of the probability of abnormal events. Utilizing a geodesic distance-based smoothing function, it effectively suppresses noise interference while maintaining spatial topological consistency, enhancing the accuracy of the probability density map in complex wharf terrain. Combined with a weighted fusion mechanism of different versions of historical inspection data, it dynamically reflects the changing trends of risk hotspots over time, improving the model's comprehensive perception of long-term evolutionary patterns and short-term emergencies. The resulting high-resolution probability density distribution map provides a scientific basis for UAV flight path planning, significantly improving the coverage density of high-risk areas and the efficiency of optimized allocation of inspection resources.

[0033] Preferably, the step of obtaining the cruise point set based on the probability density distribution map using an importance sampling algorithm further includes: Initial sampling probability distribution data is obtained based on the probability density distribution map. The sampling probability distribution data is used as the basis for decision-making in UAV inspection missions. The initial sampling probability is successively refined through a multi-level threshold grid to obtain a multi-scale sampling grid, and a constraint-aware sampling algorithm is used to obtain a set of candidate sampling points. The optimal set of cruise points is obtained by screening candidate sampling point sets through allocation strategies, calibrating the coverage of critical safety areas at the dock, and performing multi-objective optimization.

[0034] By employing an importance sampling algorithm based on probability density distribution maps, key cruise points are efficiently extracted from high-risk areas. This transforms the probability distribution of abnormal events into a decision-making basis for UAV inspection missions, enhancing the intelligence and targeting of route planning. A multi-scale sampling grid is constructed using multi-level threshold grid refinement technology, balancing global coverage with local focus capabilities to accurately identify potential risk hotspots at different spatial scales. Constraint-aware sampling algorithms are combined to introduce dock environment limitations, ensuring that the candidate point set maximizes risk area coverage while meeting safe flight requirements. Furthermore, through allocation strategy screening, safety critical area coverage calibration, and multi-objective optimization mechanisms, a comprehensive balance is struck between inspection efficiency, coverage integrity, and resource consumption. Ultimately, a reasonably distributed, low-redundancy, and highly adaptable optimal cruise point set is generated, significantly enhancing the overall effectiveness and reliability of UAV inspection missions.

[0035] Optionally, the drone autonomously performs the task of assembling patrol points, continuously collecting video and image data during flight and performing real-time analysis, immediately issuing alarms for any identified anomalies. All alarm data automatically generates inspection reports and is synchronously transmitted back to the data pool, forming a closed-loop data stream.

[0036] Preferably, the steps of collecting task data for route planning data and updating the data pool based on route planning data further include: Based on the flight path mission, the drone performs inspection missions and collects multimodal mission data in real time. The multimodal mission data includes at least image data, location data, and environmental data. Multimodal task data is filtered and updated based on geometric quality assessment, temporal quality assessment, and semantic quality assessment. Obtain the corresponding inspection quality score based on multimodal task data, and update the data pool based on multimodal task data.

[0037] By using drones to perform flight missions and collect multimodal data such as images, location, and environment in real time, comprehensive perception and information complementarity of the dock inspection site are achieved, improving data integrity and reliability. Combined with geometric quality assessment (e.g., image clarity, positioning accuracy), temporal quality assessment (e.g., continuity, consistency), and semantic quality assessment (e.g., target recognition confidence, event discrimination accuracy), the raw multimodal data is intelligently filtered and cleaned to effectively remove redundant, ambiguous, or abnormal data, ensuring data quality. Based on high-quality multimodal data, an inspection quality score is automatically generated, quantifying the execution effect of each flight mission and providing a quantitative basis for system performance monitoring. Finally, the evaluated and optimized data is dynamically updated to the data pool, forming a closed-loop iterative mechanism to continuously enrich the basic dataset, supporting subsequent model training, strategy optimization, and flight route planning upgrades, significantly enhancing the adaptive capability and intelligence level of the entire inspection system. The rule base needs to be transformed from qualitative descriptions into mathematical constraints and weights that optimization algorithms can understand. Fixed weights are assigned to different event types (e.g., fire alarm = 10, personnel intrusion = 5, illegal parking = 1). Weights can be dynamically adjusted according to factors such as time and weather. For example, in dry, windy weather, the weight of fire risk is automatically increased. In flight route cost calculations, distance to hazards such as buildings and high-voltage power lines is introduced as a penalty. The closer the distance, the higher the penalty cost, guiding flight routes away from danger.

[0038] Preferably, the step of iteratively updating the inspection strategy based on the data pool through models and rules further includes: Historical and current inspection data from the data pool are used to quantify the quality score of the current inspection strategy through an indicator system. Based on the policy evaluation results, the computer vision model and optimization algorithm are continuously optimized through online learning algorithms.

[0039] By quantifying the strategy quality score of historical and current inspection data through an indicator system, real-time monitoring and objective evaluation of inspection strategy performance are achieved. Online learning algorithms continuously optimize the computer vision model and optimization algorithms, enabling dynamic adaptation and long-term intelligent evolution of the inspection strategy. In hotspot areas, sampling is no longer uniform but based on probability distribution, with sampling points denser in high-density areas and sparser in low-density areas. High-yield areas undergo grid refinement to generate more detailed patrol points, ensuring accurate coverage of key areas.

[0040] Second Embodiment A second aspect of the present invention provides a drone route planning device for dock inspection, comprising: The map building module is used to obtain baseline map layer data based on multi-source geographic data through layered fusion and verification; The data management module is used to aggregate multi-source data into a data pool by building indexes and managing the lifecycle; The real-time analysis module is used to analyze the results data in real time based on the data pool using computer vision and streaming processing; The report generation module is used to generate inspection data based on the analysis results data using preset templates and natural language. The route planning module is used to obtain route planning data based on the event distribution hotspots and violation frequency data of the inspection data through kernel density estimation and adaptive sampling. The task update module is used to collect task data for route tasks based on route planning data and update route planning data and data pool. The strategy optimization module is used to iteratively update the inspection strategy based on the data pool through models and rules.

[0041] The system employs a map building module to achieve layered fusion and accurate verification of multi-source geographic data, generating highly reliable baseline map layer data. A data management module establishes a unified index and lifecycle mechanism, effectively integrating heterogeneous data from drones, cameras, and other sources into a dynamically updated data pool. A real-time analysis module, combining computer vision and streaming technology, efficiently analyzes real-time video streams and monitoring data, quickly identifying abnormal events and outputting alarm information. A report generation module, based on preset templates and natural language generation algorithms, automatically and structurally outputs inspection data including inspection reports and intelligent summaries, improving information presentation efficiency. A route planning module, based on event distribution hotspots and violation frequency, uses kernel density estimation and adaptive sampling algorithms to scientifically generate optimal route planning data covering high-risk areas. A task update module drives drones to perform inspection tasks, collecting multi-modal data such as images, location, and environment in real time, and feeding this data back to the data pool after quality assessment, achieving closed-loop system optimization. A strategy optimization module continuously quantitatively evaluates the effectiveness of inspection strategies based on historical and current data, iteratively optimizing computer vision models and path planning algorithms through online learning. The overall system forms a closed-loop intelligent system encompassing perception, analysis, decision-making, execution, and optimization, significantly improving the automation level, response speed, and decision-making accuracy of port inspections. This embodiment also includes several drones, specifically DJI Airport 3 paired with an M4T drone. The aforementioned system is applied at a port containing several large containers.

[0042] Drones push video streams to edge computing nodes via protocols such as RTMP or WebRTC. Edge servers use high-performance inference engines (such as NVIDIA TensorRT and Intel OpenVINO) to decode and analyze the video streams in real time. The analysis results (metadata) are fed into a complex event processing engine. This engine determines whether to trigger an alarm based on predefined rules (such as "two unknown individuals appear in the same area within 10 seconds"). Once an alarm is confirmed, the system pushes the alarm information (including location, type, and keyframes) to the monitoring center or security personnel's mobile terminals within milliseconds via a high-speed communication link (such as 5G), potentially triggering audible and visual alarms.

[0043] In the description of this application, it should be noted that the terms "inner" and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product is in use. They are used only for the convenience of describing this application and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0044] It should also be noted that, unless otherwise explicitly specified and limited, the terms "setup" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0045] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific identification content executed by the system and device described above can be referred to the corresponding process in the foregoing method embodiments.

[0046] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they shall still fall within the protection scope of the present invention.

Claims

1. A method for planning the flight path of an unmanned aerial vehicle (UAV) for dock inspection, characterized in that, include: Baseline map layer data is obtained by hierarchical fusion and verification based on multi-source geographic data; By building indexes and managing lifecycles, multi-source data can be aggregated into a data pool; Based on the data pool, the results are analyzed in real time using computer vision and streaming processing. The results include at least abnormal events and alarms. Based on the analysis results, inspection data is generated using preset templates and natural language, and the inspection data includes inspection reports and intelligent summaries. Based on the event distribution hotspots and violation frequency data of the inspection data, route planning data is obtained through kernel density estimation and adaptive sampling. Based on the route planning data, the route task collects the task data of the route task and updates the route planning data and the data pool; The inspection strategy is updated iteratively based on the data pool and the model and rules.

2. The UAV route planning method for dock inspection according to claim 1, characterized in that, The steps for obtaining baseline map layer data based on multi-source geographic data through hierarchical fusion and verification further include: Obtain map data of the dock, wherein the zoom level of the map data is larger than that of a tile, and the map data includes at least the road network and the outline of the fixed buildings of the dock; Based on the map data of the wharf, the image data collected by the UAV, and the ground control coordinate data, the baseline map layer data is obtained by extracting features and aligning features at a preset scale through the hierarchical feature pyramid network of the encoder. The initial baseline map layer data integrates geometrically consistent feature information from multiple sources. Based on the map data of the terminal and the initial reference map layer data, the optimal geometric correction parameters are obtained by iterative optimization through an adaptive transformation model with added structural constraints, and the reference map layer data is updated. The structural constraints include at least the grid structure constraints of the container yard, the linear geometric constraints of the quay crane track, and the topological connectivity constraints of the road network. A time-series dataset is constructed based on the map data of the wharf and the image data collected by UAVs. A dynamic loss function is constructed through a registration algorithm guided by change detection to adjust the network hyperparameters and update the baseline map layer data. The updated base map layer data is obtained by weighting different feature data based on different versions of the base map layer data using a probabilistic fusion algorithm.

3. The UAV route planning method for dock inspection according to claim 2, characterized in that, The calculation expression for obtaining the updated baseline map layer data by weighting and updating different feature data using a probabilistic fusion algorithm is as follows: In the formula, The location data with historical version index i is (x, y) weighted data, N is the total number of versions of the base map layer data involved in the calculation, and T is the time series length. These are the uncertainty estimates for historical version index i and position data (x, y); The calculation expression for the baseline map layer data is: In the formula, The feature values ​​are obtained by probability fusion from the location data (x, y). The feature value of the position data (x, y) with index i in the historical version.

4. The UAV route planning method for dock inspection according to claim 1, characterized in that, The step of analyzing the results data in real time using computer vision and streaming processing based on the data pool further includes: A real-time data stream is constructed using video stream data transmitted in real time from drones and monitoring data from fixed cameras at the dock. Based on the real-time data stream, the hyperparameters of the real-time inference model are updated by constructing a loss function for multi-target nodes of edge computing nodes and backpropagating the gradient of the loss function. The loss function includes target detection loss, semantic segmentation loss, and multi-target tracking loss.

5. The UAV route planning method for dock inspection according to claim 1, characterized in that, The step of obtaining route planning data based on the event distribution hotspots and violation frequency data of the inspection data through kernel density estimation and adaptive sampling further includes: Based on event location data and data from historical inspection data, a historical inspection distribution map and violation distribution data are constructed. Based on the historical inspection distribution map and the violation distribution data, a continuous probability density distribution map is generated using a multidimensional kernel density estimation algorithm. The set of cruise points is obtained based on the probability density distribution map using an importance sampling algorithm; Based on the cruise point set and the initial sampling results, the sampling accuracy is increased and the sampling results are updated using an adaptive grid subdivision algorithm; Based on the sampling results, the optimal route planning data is obtained through a constrained path optimization algorithm; The route planning data is updated by satisfying the constraints based on the constraints of the dock environment and the constraints of the UAV.

6. The UAV route planning method for dock inspection according to claim 5, characterized in that, The expression for multidimensional kernel density estimation is: In the formula, Let N be the probability density of the abnormal event at coordinates (x, y), and let N be the total number of versions of the historical inspection data used in the calculation. The different geometric features of the terminal include at least quay cranes, rails, and container yards. This is the weighted data for the violation event with version index number i, based on historical inspection data. For the feature dataset based on coordinates (x, y), For different types of geometric features of the dock, the smoothed values ​​are given in the most stable direction. This refers to the geodetic distance during inspections at the dock.

7. The method for planning unmanned aerial vehicle (UAV) routes for dock inspection according to claim 5, characterized in that, The step of obtaining the cruise point set based on the probability density distribution map using an importance sampling algorithm further includes: Initial sampling probability distribution data is obtained based on the probability density distribution map, and the sampling probability distribution data is used as the basis for decision-making in UAV inspection missions. The initial sampling probability is successively refined through a multi-level threshold grid to obtain a multi-scale sampling grid, and a constraint-aware sampling algorithm is used to obtain a set of candidate sampling points. The optimal set of cruise points is obtained by screening candidate sampling point sets through allocation strategies, calibrating the coverage of critical safety areas at the dock, and performing multi-objective optimization.

8. The UAV route planning method for dock inspection according to claim 1, characterized in that, The steps of collecting task data for the route planning data and updating the data pool based on the route planning data further include: Based on the aforementioned flight path mission, the UAV performs inspection tasks and collects multimodal mission data in real time. The multimodal mission data includes at least image data, location data, and environmental data. The multimodal task data is filtered and updated based on geometric quality assessment, temporal quality assessment, and semantic quality assessment. The corresponding inspection quality score is obtained based on the multimodal task data, and the data pool is updated based on the multimodal task data.

9. The method for planning unmanned aerial vehicle (UAV) routes for dock inspection according to claim 1, characterized in that, The step of iteratively updating the inspection strategy based on the data pool through models and rules further includes: Historical and current inspection data from the data pool are used to quantify the quality score of the current inspection strategy through an indicator system. Based on the policy evaluation results, the computer vision model and optimization algorithm are continuously optimized through online learning algorithms.

10. A drone route planning device for dock inspection, characterized in that, include The map building module is used to obtain baseline map layer data based on multi-source geographic data through layered fusion and verification; The data management module is used to aggregate multi-source data into a data pool by building indexes and managing the lifecycle; The real-time analysis module is used to analyze the results data in real time based on the data pool using computer vision and streaming processing. The report generation module is used to generate inspection data based on the analysis results data using preset templates and natural language. The route planning module is used to obtain route planning data based on the event distribution hotspots and violation frequency data of the inspection data through kernel density estimation and adaptive sampling. The task update module is used to collect task data of the route task based on the route planning data, update the route planning data, and update the data pool; The strategy optimization module is used to update the inspection strategy based on the data pool through iterative updates of models and rules.

Images