Port unmanned aerial vehicle collaborative operation system and method based on digital twinning and AI

Abstract

The invention discloses a port unmanned aerial vehicle collaborative operation system and method based on digital twin and AI. The method comprises the following steps: collecting multi-source heterogeneous data of an unmanned aerial vehicle, a fixed sensor and a service system in a port in real time; dynamically and synchronously updating the digital twinborn model of the port based on the multi-source heterogeneous data; dividing a port physical space into three-dimensional voxels, and calculating and updating a dynamic risk potential field representing port global risk distribution in real time based on a digital twin model; based on the dynamic risk potential field, identifying a high-potential risk area, and dynamically generating an unmanned aerial vehicle task for the high-potential risk area; and scheduling the unmanned aerial vehicle to execute the unmanned aerial vehicle task, and analyzing the unmanned aerial vehicle return data to perform closed-loop verification. According to the method, a feed-forward risk prediction mechanism is constructed, a traditional passive response mode is converted into active intervention, the response delay of port abnormal events is shortened, intelligent scheduling of monitoring resources is achieved, and the safety and efficiency of port operation are improved.

Description

Technical Field

[0001] This application relates to the field of port automation management and safety monitoring technology, and in particular to a port drone collaborative operation system and method based on digital twins and artificial intelligence. Background Technology

[0002] With the continued surge in global trade volume, ports, as core hubs of the international logistics network, are facing unprecedented challenges in terms of operational efficiency, security, and intelligence. Current port management and monitoring systems still largely rely on fixed-location closed-circuit television (CCTV) systems, periodic manual inspections, and some single-function, independent drone applications. These traditional methods suffer from numerous shortcomings, including numerous blind spots, limited perception dimensions, severe data silos, and delayed emergency response. They fail to provide a comprehensive and dynamic understanding of the overall port situation, especially in responding to emergencies, often resulting in a passive "post-event response," potentially missing optimal handling opportunities and causing unnecessary economic losses and safety hazards. Therefore, the market urgently needs a new technological solution that can deeply integrate the physical world and cyberspace to achieve proactive perception, intelligent decision-making, and efficient collaboration to address these technical problems. Summary of the Invention

[0003] The purpose of this application is to provide a port drone collaborative operation system and method based on digital twins and AI, which aims to solve the technical problems of long response delay, incomplete perception and weak collaborative ability in existing port monitoring technologies.

[0004] Firstly, this application provides a method for collaborative operation of port drones based on digital twins and artificial intelligence (AI), comprising: real-time acquisition of multi-source heterogeneous data within the port, the multi-source heterogeneous data including dynamic perception data transmitted back by a drone swarm, static perception data collected by port fixed facilities, and business data generated by the port business system; dynamically and synchronously updating the port's digital twin model based on the multi-source heterogeneous data; constructing a dynamic risk potential field characterizing the spatiotemporal distribution of risks across the entire port area, the construction of the dynamic risk potential field comprising: dividing the physical space of the port into three-dimensional voxels of a preset size, and calculating the risk potential value of each three-dimensional voxel in real time based on entity attributes and dynamic events in the digital twin model; dynamically generating drone tasks based on the dynamic risk potential field, the generation of drone tasks comprising: identifying areas where the risk potential value exceeds a preset risk threshold as high-potential risk areas, and generating corresponding drone inspection or detailed investigation tasks for the high-potential risk areas; scheduling drones in the drone swarm to execute the drone tasks, and analyzing the data transmitted back by the drones executing the tasks to achieve closed-loop verification.

[0005] Optionally, the step of calculating the risk potential value of each three-dimensional voxel in real time includes: determining a basic static risk value for each three-dimensional voxel based on the invariant or slowly changing properties of the entity in the digital twin model; monitoring dynamic events in the digital twin model in real time, and calculating the dynamic risk contribution value of the dynamic event to the three-dimensional voxel according to the risk intensity of the dynamic event and its spatial distance from the three-dimensional voxel; and fusing the basic static risk value and the dynamic risk contribution value to obtain the final risk potential value of the three-dimensional voxel.

[0006] Optionally, the step of fusing the basic static risk value and the dynamic risk contribution value further includes: acquiring real-time environmental meteorological data and determining an environmental correction coefficient based on the environmental meteorological data; and using the environmental correction coefficient to correct the dynamic risk contribution value to reflect the impact of environmental factors on risk propagation and evolution.

[0007] Optionally, the risk intensity of the dynamic event depends on the type of the dynamic event, which includes quay crane lifting operations, automated guided vehicle (AGV) operation, ship berthing or unberthing operations, and personnel entering a specific area.

[0008] Optionally, the step of dynamically generating drone missions further includes: selecting suitable drone types and mission payloads from the drone swarm based on the risk cause type of the high-potential risk area; and planning an optimal flight path for the selected drone that starts from its current location or automated pod, reaches the high-potential risk area, and avoids dynamic and static obstacles.

[0009] Optionally, the drone type includes wide-area inspection drones, precision detection drones, and environmental monitoring drones; the mission payload includes high-definition visible light gimbals, infrared thermal imaging gimbals, lidar, gas sensors, and water quality sensors.

[0010] Optionally, the step of analyzing the data transmitted back by the drone performing the task to achieve closed-loop verification includes: using AI visual analysis algorithms to analyze the video or image data transmitted back by the drone in real time to identify whether there are any abnormalities in the appearance or state of physical entities; if an abnormality is identified, the emergency response process is automatically triggered, and the type of the abnormality, its three-dimensional spatial location, the real-time video link, and the system-planned safe approach route are pushed to the ground emergency response terminal.

[0011] Secondly, this application provides a port drone collaborative operation system based on digital twins and artificial intelligence (AI), comprising: a multi-source heterogeneous data acquisition module configured to collect in real time dynamic perception data transmitted back by a fleet of drones within the port, static perception data collected by fixed port facilities, and business data generated by the port's business system; a digital twin synchronization module configured to dynamically and synchronously update the port's digital twin model based on the multi-source heterogeneous data; and a risk potential field calculation module configured to construct a dynamic risk potential field characterizing the spatiotemporal distribution of risks across the entire port area, specifically configured to divide the port's physical space into preset sizes. The system generates three-dimensional voxels and calculates the risk potential value of each voxel in real time based on entity attributes and dynamic events in the digital twin model. An intelligent task planning and scheduling module is configured to identify areas where the risk potential value exceeds a preset risk threshold as high-potential risk zones based on the dynamic risk potential field, and dynamically generate corresponding UAV inspection or detailed investigation tasks for these high-potential risk zones. The intelligent task planning and scheduling module is also configured to schedule UAVs in a UAV cluster to execute the UAV tasks. An AI analysis platform is configured to analyze the data transmitted back by the UAVs executing the tasks to achieve closed-loop verification.

[0012] Optionally, the risk potential field calculation module is specifically configured to: determine a basic static risk value for each three-dimensional voxel based on the invariant or slowly varying properties of entities in the digital twin model; monitor dynamic events in the digital twin model in real time, and calculate the dynamic risk contribution value of the dynamic events to the three-dimensional voxels based on the risk intensity of the dynamic events and their spatial distance from the three-dimensional voxels; and fuse the basic static risk value and the dynamic risk contribution value to obtain the final risk potential value of the three-dimensional voxels. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart illustrating a port drone collaborative operation method based on digital twins and AI, provided in one embodiment of this application.

[0015] Figure 2 This is a schematic diagram of the dynamic risk potential field calculation process provided in one embodiment of this application.

[0016] Figure 3This is a schematic diagram of the structure of a port drone collaborative operation system based on digital twins and AI, provided in one embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0019] Example 1 This embodiment provides a collaborative operation method for port drones based on digital twins and artificial intelligence (AI), referring to... Figure 1 The specific process of this method may include the following steps: S100: Real-time collection of multi-source heterogeneous data within the port, including dynamic perception data transmitted back by UAV swarms, static perception data collected by port fixed facilities, and business data generated by the port business system.

[0020] In this step, the system constructs a comprehensive port-wide perception network, serving as the data foundation for all subsequent analysis and decision-making. The purpose of this network is to break down data silos in traditional monitoring systems and achieve digital mapping of all elements of the port, including "people, machines, cargo, and yards."

[0021] The dynamic sensing data transmitted back by drone swarms is the most flexible and proactive component of the sensing network. Drone swarms can include various types of drones to adapt to different mission requirements. For example, long-endurance fixed-wing drones can be deployed for large-scale, periodic wide-area patrols to acquire macroscopic images of the entire port area; quadcopter drones equipped with high-magnification zoom cameras and infrared thermal imagers can be deployed for close-range, detailed structural detection of tall equipment such as quay cranes and yard cranes; and environmental monitoring drones equipped with specific gas sensors (such as chlorine and hydrogen sulfide sensors) or water quality samplers can be deployed for targeted monitoring of key areas such as hazardous materials storage yards and sewage outlets. These drones transmit the collected high-definition video streams, infrared thermal images, lidar point cloud data, and sensor readings back to the ground data center in real time via onboard 5G or 4G communication modules.

[0022] Static sensing data collected by port fixed facilities mainly includes a closed-circuit television (CCTV) network covering the port area, as well as Internet of Things (IoT) sensors deployed in key equipment or areas. The CCTV network provides continuous video surveillance of main channels, yards, and berths. IoT sensors can provide richer status information, such as temperature sensors installed on refrigerated containers, vibration and temperature sensors installed on key bearings of quay cranes, and wind speed, wind direction, temperature, and humidity data collected by weather stations deployed in the yard.

[0023] The business data generated by the port's operational system mainly comes from the Terminal Operating System (TOS) and the Automatic Identification System (AIS). TOS data provides the precise location, container number, type, weight, and information on the cargo loaded inside each container (especially whether it contains dangerous goods). AIS data provides real-time information on the location, speed, course, vessel type, and destination of vessels entering and leaving the port.

[0024] The system uses a unified data access platform and multiple protocols such as MQTT, RTSP, and HTTP / 2 to collect data from all the aforementioned sources in real time. The collected data undergoes preliminary preprocessing, including timestamp alignment, coordinate system registration (e.g., converting all data to the WGS-84 geographic coordinate system or the port's custom local coordinate system), and data format normalization, in preparation for subsequent data fusion and model synchronization.

[0025] S200: Based on the aforementioned multi-source heterogeneous data, dynamically and synchronously update the digital twin model of the port.

[0026] In this step, the system uses the data collected and fused in S100 to update the pre-built port digital twin model in real time and dynamically, so as to achieve accurate and synchronous mapping of virtual space to physical space.

[0027] Upon initial deployment, this digital twin model can be constructed by performing a comprehensive 3D scan of the entire port using an oblique photography camera and LiDAR mounted on a drone. This generates a basic geographic information model of the port with realistic textures and centimeter-level accuracy. This basic model includes all static infrastructure such as wharves, shorelines, storage yards, roads, and buildings. In a preferred embodiment, the digital twin model is centimeter-level accurate to ensure a high-fidelity spatial basis for subsequent risk assessments and drone path planning.

[0028] In daily operations, the data streams collected by the S100 drive millisecond-level updates to dynamic elements in the model. For example, when a container truck or AGV is moving through the port area, the location data transmitted back by its onboard GPS or differential positioning system updates its 3D position and orientation in the digital twin model in real time. When the TOS system records a container transfer operation, the container's position in the digital twin model moves precisely from one stacking location to another. When AIS data shows a cargo ship approaching a berth, its virtual vessel in the model synchronously performs a berthing maneuver. When thermal imaging data transmitted from a drone shows that the motor temperature of a device is rising, the corresponding component of that motor in the model is rendered with a warning color and accompanied by a real-time temperature reading.

[0029] In this way, the digital twin model becomes a single source of truth, aggregating all information. It not only replicates the geometry of the physical port but also the physical processes and business logic that occur within it. This provides a highly realistic virtual experimental environment for subsequent complex analysis, simulation, and risk prediction.

[0030] S300: Construct a dynamic risk potential field that characterizes the spatiotemporal distribution of risk across the entire port area. The construction of the dynamic risk potential field includes: dividing the physical space of the port into three-dimensional voxels of a preset size, and calculating the risk potential value of each three-dimensional voxel in real time based on the entity attributes and dynamic events in the digital twin model.

[0031] This step introduces a feedforward-based risk assessment model based on physical field theory. Its purpose is to transform the massive, discrete, and heterogeneous state data carried by the digital twin model in S200 into a unified indicator with predictive capabilities that can intuitively and continuously characterize the safety situation of the entire port—the risk potential field.

[0032] To perform continuous spatial calculations, the system discretizes the entire port's three-dimensional physical space into closely spaced, uniformly sized cubic mesh cells, or three-dimensional voxels. The voxel size is a configurable parameter that determines the spatial resolution of the risk assessment. For example, the voxel size can be set to... This means that every cubic meter of space in the port will have an independent risk value that is calculated in real time.

[0033] Then, the system calculates the risk potential value of each 3D voxel in real time and in parallel. In a preferred embodiment, any voxel... At any moment Total risk potential This can be determined through a comprehensive mathematical model. This model quantifies and superimposes multiple risk factors. Its core mathematical form can be expressed as: in, Represents voxels The inherent risk level of a location, which does not change over time or only changes slowly over time, reflects the "background" risk of the area. This value is typically calibrated during system initialization based on an expert knowledge base and static planning data of the port. For example, different static risk values ​​can be assigned to voxels of different functional areas within the port. For instance: For voxels located in hazardous chemical storage areas (such as liquid chlorine and fuel oil tank areas), It can be set to a relatively high value, such as 0.8-0.9.

[0034] For voxels within a typical container yard, It can be set to a medium value, such as 0.4-0.5.

[0035] For operating areas covered by large special equipment such as quay cranes and yard cranes, It can be set to 0.6-0.7.

[0036] For areas such as office buildings and main roads in the port area, It can be set to a lower value, such as 0.1-0.2.

[0037] These values ​​are stored in a static lookup table corresponding to the voxel grid.

[0038] This is the most crucial and dynamic part of the risk potential field model, capturing in real time the instantaneous risks arising from various operational events occurring within the port. Its core idea is that any dynamic event... All will occur at their point of occurrence A risk field is generated, the influence of which decreases with increasing distance from the event's origin. (Vollometer) The dynamic risk contribution value is the linear sum of the impacts of all dynamic events occurring at that moment. Its calculation formula is: in: It is an event The inherent risk intensity is a dimensionless numerical value used to characterize the degree of danger of different types of events. This value can be dynamically determined based on the event type and operational parameters (such as speed and weight). For example: The quay crane lifted a 40-foot container marked as dangerous goods. The value could be as high as 0.9.

[0039] An AGV is traveling at a speed of 10 m / s at an intersection. The value could be 0.7.

[0040] A large oil tanker is currently berthing. The value could be 0.8.

[0041] A worker entered an unauthorized hazardous materials storage area. The value could be 0.95.

[0042] It is a voxel Central point and event Central voxel of occurrence The three-dimensional Euclidean distance between them, in meters.

[0043] This is a risk intensity decay function with distance, used to simulate the localized impact of risk. This function must be a monotonically decreasing function, and... The value is 1. Various function forms are possible; for example, the Gaussian decay function can be used: in, This is a parameter representing the range of risk impact, measured in meters. It controls the rate at which the risk decays. The larger the value, the wider the impact of the event and the slower its decay. Different events can have different values. Value. For example, the physical impact range of a single hoisting operation is limited, its The value can be set to 20 meters; however, a potential chemical spill can have a wide-ranging impact. The value can be set to 100 meters.

[0044] This coefficient is used to quantify the corrective effect of real-time environmental factors on risks, particularly gas diffusion and fire spread. It is a dimensionless coefficient. For example, for chemical spill risk, this coefficient could be a function of wind speed and direction. For instance, a real-time wind speed vector could be obtained from a weather station. For a risk source Pointing voxels displacement vector It can calculate the angle between the wind direction and the direction of displacement. When the wind direction is the same as the displacement direction (downwind), the risk diffusion effect is enhanced. It should be greater than 1; when the wind direction is opposite to the displacement direction (against the wind), the risk diffusion effect is weakened. It should be less than 1. A simplified model could be: in It is an adjustable sensitivity coefficient. Under risk types with no environmental impact or in windless weather, The possible value is 1.

[0045] and It is a weighting coefficient used to adjust the proportion of static risk and dynamic risk in the overall risk assessment. It is dimensionless and satisfies... These two coefficients can be dynamically adjusted based on the overall operational status of the port. For example, during nighttime or periods of low activity, static risks (such as aging facilities or unauthorized intrusion) dominate, and the coefficients can be increased accordingly. The value. During peak daytime work periods, dynamic work risks occur frequently, at which time the value can be increased. The value of .

[0046] Through the steps described above, the system can utilize parallel computing resources such as GPUs to perform high-speed, parallel risk potential calculations on millions of voxels in the port space, thereby generating and updating a three-dimensional, dynamic port risk potential field map in real time. This map visually reveals which areas of the port have lower security barriers and are more prone to security incidents at the current moment.

[0047] S400: Based on the dynamic risk potential field, dynamically generate UAV tasks. The generation of UAV tasks includes: identifying areas where the risk potential value exceeds a preset risk threshold as high-potential risk areas, and generating corresponding UAV inspection or detailed investigation tasks for the high-potential risk areas.

[0048] In this step, the system uses the risk potential field map generated by S300 as the decision input, transforming dynamic monitoring resources such as UAVs from the traditional fixed-route inspection mode to a risk-driven, on-demand intelligent scheduling mode.

[0049] The system will continuously scan the entire three-dimensional matrix of the risk potential field, and retrieve all risk potential values. Exceeding the preset risk threshold Connected regions were identified and defined as "high-potential risk areas". Risk threshold. It is a parameter that can be dynamically adjusted according to security management requirements. For example, multiple threshold levels can be set, such as... (Attention level) (Alarm level)

[0050] Once a high-risk area is identified, the intelligent mission planning and scheduling module is immediately activated, automatically generating an optimal drone mission for that area. This process includes risk attribution and mission decision-making, as well as drone and payload selection.

[0051] The system will trace back to the main risk sources that constitute this high-potential-risk area. This is done by analyzing the factors that contribute the most to the risk. and This is achieved through specific items. For example, if the risk is primarily generated by dynamic events of a particular device (such as abnormal motor temperature reflected in the digital twin model), thus resulting in high risk... If the risk is mainly contributed by the static high-risk value of the hazardous materials storage yard, the system will determine the risk type as "equipment failure risk" and generate a "detailed equipment inspection" task. If the risk is mainly contributed by the static high-risk value of the hazardous materials storage yard, the system will generate a "routine inspection of the hazardous materials storage yard" task.

[0052] Based on the determined risk type, the system selects the most suitable drone and sensor payload from the drone resource pool. For example: For "equipment failure risks," especially those related to temperature, the system will dispatch a quadcopter drone equipped with a high-magnification zoom and infrared thermal imaging gimbal to conduct close-range, non-contact temperature measurement and detailed observation.

[0053] For the risk of "chemical spills", the system will prioritize dispatching an environmental monitoring drone equipped with the corresponding chemical sensor.

[0054] For a large, high-risk area with unknown causes, the system can first dispatch a wide-area inspection drone for rapid approach and macroscopic reconnaissance.

[0055] Optimal Path Planning: After determining the UAV and the target point, the system plans an optimal flight path for the UAV in the three-dimensional space defined by the port's digital twin model. This path planning algorithm (such as the A algorithm or the RRT algorithm) considers multiple factors: Efficiency: Select the path with the shortest distance or the shortest flight time.

[0056] Safety: The path must be able to automatically avoid all static obstacles (such as buildings and quay cranes) and dynamic obstacles (such as other drones and vehicles in operation) marked in the digital twin model, and comply with preset electronic fence and no-fly zone rules.

[0057] Mission-oriented: The endpoint of the path is not only a three-dimensional coordinate, but may also include a specific observation angle and hovering position to ensure that the mission payload can detect the target from the best perspective.

[0058] S500: Schedule drones in the drone cluster to execute the drone mission, and analyze the data returned by the drones executing the mission to achieve closed-loop verification.

[0059] In this step, the system implements the tasks planned in S400 and performs intelligent analysis on the execution results, thus forming a complete closed loop from "risk discovery" to "event confirmation".

[0060] Mission scheduling instructions are sent to the automated drone airfield (drone) and the corresponding drone flight control system. Upon receiving the instructions, the automated airfield automatically opens its hatch, pushes the drone onto the takeoff platform, and performs rapid charging or battery replacement. After receiving the mission package (containing waypoints, mission actions, etc.), the drone automatically takes off and strictly follows the planned path to the high-risk area.

[0061] During the mission, the drone transmits data collected by its sensors (such as high-definition video streams) back to the AI ​​analysis platform in real time via a 5G network. The AI ​​analysis platform deploys a series of visual analysis algorithm models to process the transmitted data in real time.

[0062] For example, this application provides a specific implementation of the AI ​​visual analysis algorithm model, but this should not be construed as the only limitation of this application.

[0063] The target detection and anomaly recognition model can be implemented using a Transformer-based end-to-end target detection architecture (DETEction TRansformer, DETR). The model's input is data transmitted from the drone's camera, with dimensions normalized to, for example... The image is an RGB image of pixels. The image is first processed through a convolutional neural network (CNN) backbone (e.g., ResNet-50) to extract low-level visual feature maps. These feature maps are then flattened and combined with positional encodings to form a feature sequence, which is input into a standard Transformer encoder-decoder architecture. The encoder captures global contextual information of the image through a self-attention mechanism, while the decoder decodes in parallel from the encoder's output a predetermined number (e.g., 100) of "object queries." Each decoded result is a vector containing a class prediction and bounding box coordinates. The model's output is a set containing all detected objects in the image, each with its class label (e.g., "container crack," "equipment rust," "helmet not worn") and a corresponding confidence score. To ensure accurate application of the model to port scenes, it needs to be supervisedly trained on a large-scale, finely annotated port image dataset. This dataset should contain millions of image samples of port equipment, containers, and personnel, covering various weather conditions, lighting, and shooting angles. By minimizing the category classification loss and bounding box regression loss between the predicted results and the true annotations, the model can learn an accurate representation of port-specific anomaly patterns. In this way, those skilled in the art can explicitly construct and train an AI analysis model that meets the requirements of this application without inventive effort.

[0064] The analysis results will lead to two follow-up processes: If the AI ​​analysis platform confirms the existence of a genuine anomaly (for example, a crack is clearly identified in the video on the container, and the chlorine sensor reading transmitted from the environmental monitoring drone shows a slight exceedance), the system will immediately escalate the alert level from "high potential risk" to "confirmed safety incident." Subsequently, the system will automatically trigger the emergency response process. This includes: On the 3D interface of the human-computer interaction and visualization module, the container is highlighted in red as a warning, and an event details window pops up. Alarm information is automatically pushed to the port emergency response team's mobile terminals (such as mobile apps and walkie-talkies). This information package includes: the event type (e.g., "Level 1 chemical leak"), precise 3D geographic coordinates, a link to the real-time video stream, information on the leaked substance (liquid chlorine), and a safe approach route for ground personnel and vehicles automatically planned by the system based on real-time wind field data and road conditions.

[0065] If, after repeated and thorough drone searches of a high-risk area, neither the AI ​​analysis platform nor human review reveals any genuine anomalies, the alert is marked as a "benign false alarm." The data for this event (including the risk potential field distribution that triggered the alert, related dynamic events, and on-site images ultimately confirmed to be anomalies) will be stored in a database. This data can be used as negative samples to refine parameters of the risk potential field model (such as risk intensity). Value, scope of influence The risk model can be trained and optimized offline using machine learning to reduce the false positive rate in similar scenarios in the future, enabling it to iterate and evolve on its own.

[0066] Example 2 This embodiment provides a port drone collaborative operation system based on digital twins and artificial intelligence (AI), referring to... Figure 3 This system serves as the hardware and functional carrier of the aforementioned method, and its specific structure may include: The multi-source heterogeneous data acquisition module 801 is the system's perception layer. In terms of hardware, it includes IoT gateways deployed throughout the port area, video encoders, and communication base stations and servers for receiving wireless data from drones, AIS, etc. In terms of software, it includes drivers and data adapters for parsing different data protocols (MQTT, RTSP, NMEA, etc.). Its function is to perform the data acquisition and preprocessing tasks described in S100.

[0067] The digital twin synchronization module 802 can be a high-performance digital twin engine server. This server maintains a 3D model database of the port and runs a state synchronization service. This service subscribes to all data streams published by the data acquisition module and updates the attributes of the corresponding digital assets in the model database in real time based on the data content to execute the synchronization task described in S200.

[0068] The risk potential field calculation module 803 is the system's computing unit. In terms of hardware, it typically consists of one or more servers equipped with high-end graphics processing units (GPUs) to utilize the GPU's parallel computing capabilities. In terms of software, it runs the core risk potential field calculation algorithm program of this application (which can be written in CUDA or OpenCL). This module obtains full state data from the digital twin synchronization module, performs the risk potential field calculation described in S300, and outputs the result (a three-dimensional risk matrix) to the task planning module.

[0069] The intelligent task planning and scheduling module 804 receives risk potential field data, executes the task generation algorithm described in S400 (including risk attribution, resource matching, and path planning), and sends the finally generated task instructions to the UAV flight control system and the automated nest control system through the communication interface.

[0070] Drone swarms and automated nests (not shown separately in the diagram): As the execution layer of the system, it includes various types of drones, their respective mission payloads, and automated nests that can realize automatic take-off and landing, charging / battery swapping, and data download of drones.

[0071] The AI ​​analysis platform 805 is the system's cognition and verification unit. It is usually deployed on a GPU server similar to the risk potential field calculation module. It runs various deep learning models (such as DETR, ResNet, etc.) and sensor data analysis algorithms. It receives real-time data streams transmitted back by the drones performing the tasks and performs the intelligent analysis and closed-loop verification functions described in S500.

[0072] The Human-Computer Interaction and Visualization Module 806 is a unified interface provided to management personnel. It can be a 3D visualization application running on a control center screen or engineer's workstation. This software can render digital twin models in real time, overlaying risk potential fields as heatmaps or isosurfaces onto a 3D scene. It can play real-time video transmitted from any drone in picture-in-picture mode and prominently display and manage alarm information generated by the AI ​​analysis platform. Simultaneously, it provides operators with necessary manual intervention permissions, such as manually selecting drones and defining inspection areas.

[0073] All of the above modules are interconnected and work together through the high-speed local area network within the port, forming the port UAV collaborative operation system described in this application.

[0074] It should be noted that the scope of protection of this application is not limited to the details of the above embodiments. Those skilled in the art, inspired by this application, can make various combinations, modifications, or equivalent substitutions to the above embodiments, and all such changes should be covered within the scope of protection of this application. For example, the calculation formula for the risk potential field can employ a more complex nonlinear fusion model; path planning can introduce optimization objectives that consider energy consumption; AI analysis is not limited to vision but can also integrate multimodal information such as sound and vibration. These are all natural extensions of the technical ideas of this application.

[0075] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware.

[0076] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A port unmanned machine collaborative operation method based on digital twinning and artificial intelligence (AI), characterized in that, The method comprises the following steps: Real-time acquisition of multi-source heterogeneous data in the port, including dynamic perception data returned by a UAV fleet, static perception data collected by fixed facilities in the port, and business data generated by port business systems; Based on the multi-source heterogeneous data, dynamically and synchronously update the digital twin model of the port; Construct a dynamic risk potential field representing the spatiotemporal distribution of global risks in the port, which includes dividing the physical space of the port into three-dimensional voxels of a preset size, and based on the entity attributes and dynamic events in the digital twin model, real-time calculation of the risk potential value of each three-dimensional voxel; Based on the dynamic risk potential field, dynamically generate UAV tasks, which includes identifying areas with risk potential values exceeding a preset risk threshold as high-potential-risk areas, and generating corresponding UAV inspection or detailed investigation tasks for the high-potential-risk areas; Dispatch UAVs in the UAV fleet to execute the UAV tasks, and analyze the data returned by the UAVs executing the tasks to achieve closed-loop verification.

2. The method of claim 1, wherein, The step of real-time calculation of the risk potential value of each three-dimensional voxel includes: Based on the invariable or slowly changing attributes of entities in the digital twin model, determine a basic static risk value for each three-dimensional voxel; Real-time monitoring of dynamic events in the digital twin model, and according to the risk intensity of the dynamic events and their spatial distance from the three-dimensional voxels, calculate the dynamic risk contribution value of the dynamic events to the three-dimensional voxels; Fuse the basic static risk value and the dynamic risk contribution value to obtain the final risk potential value of the three-dimensional voxel.

3. The method of claim 2, wherein, The step of fusing the basic static risk value and the dynamic risk contribution value further includes: Obtain real-time environmental meteorological data, and determine an environmental correction coefficient based on the environmental meteorological data; Use the environmental correction coefficient to correct the dynamic risk contribution value to reflect the influence of environmental factors on risk propagation and evolution.

4. The method of claim 2, wherein, The risk intensity of the dynamic event depends on the type of the dynamic event, which includes shore crane lifting operation, AGV driving, ship berthing or unberthing operation, and personnel entering a specific area.

5. The method of claim 1, wherein, The step of dynamically generating UAV tasks further includes: According to the risk cause type of the high-potential-risk area, select the appropriate UAV type and task payload from the UAV fleet; Plan an optimal flight path for the selected UAV to reach the high-potential-risk area from the current location or an automated nest, which can avoid dynamic and static obstacles.

6. The method of claim 5, wherein, The UAV types include wide-area inspection UAVs, fine-detection UAVs, and environmental monitoring UAVs; the task payloads include high-definition visible light gimbals, infrared thermal imaging gimbals, laser radars, gas sensors, and water quality sensors.

7. The method of claim 1, wherein, The step of analyzing the data returned by the UAVs executing the tasks to achieve closed-loop verification includes: Using AI visual analysis algorithms, real-time analysis of video or image data returned by the UAVs to identify whether there are appearance abnormalities or state abnormalities of physical entities; If an anomaly is identified, an emergency response process is automatically triggered, and the type of the anomaly, three-dimensional spatial position, real-time video link, and system-planned safe approach route are pushed to a ground emergency response terminal.

8. A port unmanned machine collaborative operation system based on digital twinning and artificial intelligence (AI), characterized in that, The method comprises: a multi-source heterogeneous data collection module configured to collect dynamic perception data returned by a drone fleet in a port, static perception data collected by fixed facilities in the port, and business data generated by a port business system in real time; a digital twin synchronization module configured to dynamically synchronize and update a digital twin model of the port based on the multi-source heterogeneous data; a risk potential field calculation module configured to construct a dynamic risk potential field representing the global risk spatio-temporal distribution of the port, specifically configured to divide the physical space of the port into three-dimensional voxels of a preset size, and based on entity attributes and dynamic events in the digital twin model, to calculate the risk potential value of each three-dimensional voxel in real time; an intelligent task planning and scheduling module configured to identify a region with a risk potential value exceeding a preset risk threshold as a high-potential risk area based on the dynamic risk potential field, and dynamically generate corresponding drone inspection or detailed investigation tasks for the high-potential risk area; wherein the intelligent task planning and scheduling module is further configured to schedule drones in a drone cluster to execute the drone tasks; and an AI analysis platform configured to analyze data returned by the drones executing the tasks to achieve closed-loop verification.

9. The system of claim 8, wherein, The risk potential field calculation module is specifically configured to: determine a basic static risk value for each three-dimensional voxel based on the invariable or slowly changing attributes of entities in the digital twin model; monitor dynamic events in the digital twin model in real time, and calculate a dynamic risk contribution value of the dynamic events to the three-dimensional voxels according to the risk intensity of the dynamic events and their spatial distance from the three-dimensional voxels; fuse the basic static risk value and the dynamic risk contribution value to obtain the final risk potential value of the three-dimensional voxels.

10. The system of claim 8, wherein, The step of analyzing data returned by the drones executing the tasks to achieve closed-loop verification comprises: using AI visual analysis algorithms to analyze video or image data returned by the drones in real time to identify whether there are appearance abnormalities or state abnormalities of physical entities; if an anomaly is identified, an emergency response process is automatically triggered, and the type of the anomaly, three-dimensional spatial position, real-time video link, and system-planned safe approach route are pushed to a ground emergency response terminal.

Images