Port unmanned aerial vehicle artificial intelligence scheduling method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a port unmanned aerial vehicle artificial intelligence scheduling method and system, electronic equipment and a storage medium, and relates to the technical field of port unmanned aerial vehicle scheduling technologies. The method comprises the following steps: acquiring port information and unmanned aerial vehicle information; determining a flying area and a no-fly area according to the port information; constructing a time sequence state vector according to the port dynamic elements, and predicting the time sequence state vector to obtain dynamic obstacle distribution information in a future preset step number; constructing a task priority function based on the port business characteristics, and determining the task priority of each port task according to the task priority function; determining an endurance verification result according to the flight performance of the unmanned aerial vehicle; and generating a multi-unmanned aerial vehicle scheduling sequence based on the task priority, the endurance verification result and the flight area, and executing a multi-unmanned aerial vehicle scheduling operation based on the multi-unmanned aerial vehicle scheduling sequence. According to the invention, the problem of low unmanned aerial vehicle scheduling precision is solved, and the effect of improving the unmanned aerial vehicle scheduling precision is achieved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of port drone scheduling, specifically to a port drone artificial intelligence scheduling method, system, electronic device and storage medium. Background Technology

[0002] Modern port operation areas are characterized by large volumes of business, dense equipment, and complex dynamic environments. Within the port space, numerous quay cranes, rail-mounted gantry cranes, container trucks, and densely stacked containers create a highly enclosed and dynamic physical environment. Traditional drone operation methods primarily rely on manual operation or semi-automatic control. When faced with strong micro-weather disturbances within the port, frequently changing dynamic obstacles (such as the extension and retraction of crane beams and the movement of truck groups), and high-intensity, multi-concurrent task demands, these methods suffer from technical bottlenecks such as slow response speed, high flight safety risks, and low task coordination efficiency.

[0003] Existing drone dispatching systems are mostly designed for inspections in open areas or simple, single-task operations, lacking the ability to adapt to the complex three-dimensional structure and dynamic operational rhythm of ports. Specifically, current technologies cannot effectively handle communication obstruction and spatial limitations caused by changes in container stacking height, nor can they dynamically adjust task priorities based on business events such as ship berthing and loading / unloading schedules. Furthermore, they lack the ability to predict and mitigate risks specific to port-specific wind environments (such as sea breezes and wind shear). Therefore, how to build a drone dispatching system that can deeply adapt to the dynamic environment of ports and achieve high safety, high efficiency, and intelligence is a key issue that urgently needs to be addressed in the current digital transformation of ports. Summary of the Invention

[0004] This invention provides a method, system, electronic device, and storage medium for scheduling port drones using artificial intelligence, in order to at least solve the problem of low obstacle avoidance accuracy caused by the highly dynamic port environment in related technologies.

[0005] According to an embodiment of the present invention, a port unmanned aerial vehicle (UAV) artificial intelligence scheduling method is provided, comprising: Acquire port information and drone information, wherein the port information includes port business characteristics, port dynamic elements, and port tasks, and the drone information includes drone flight performance. Based on the port information, the permitted and prohibited flight areas are determined. A time-series state vector is constructed based on the port's dynamic elements, and the time-series state vector is predicted to obtain dynamic obstacle distribution information within a preset number of steps in the future. Based on the port business characteristics, a task priority function is constructed, and the task priority of each port task is determined according to the task priority function; The endurance verification result is determined based on the flight performance of the UAV. Based on the task priority, endurance verification results, and flyable areas, a multi-UAV scheduling sequence is generated, and multi-UAV scheduling operations are executed based on the multi-UAV scheduling sequence.

[0006] In one exemplary embodiment, the method further includes: Based on preset constraints, a three-dimensional trajectory planning result is generated, and the UAV is replanned in real time according to the dynamic obstacle distribution information.

[0007] In one exemplary embodiment, the method further includes: Real-time control judgments are made based on real-time monitored wind speed thresholds, sudden dynamic obstacle status, and changes in no-fly zones to obtain real-time judgment results; as well as, Based on real-time monitored wind speed thresholds, time-series predictions of wind shear are performed to obtain flight stability protection strategies.

[0008] In one exemplary embodiment, after obtaining the port information, the method further includes: A communication probability field is constructed based on the port information, and the signal shielding zone and electromagnetic interference zone generated by the metal container group are determined through the communication probability field. Cross-modal feature fusion is performed on image and audio information collected by UAVs, and the location and trajectory prediction of non-line-of-sight moving targets are performed.

[0009] In one exemplary embodiment, constructing the communication probability field based on the port information includes: Obtain the reference signal received power, signal-to-noise ratio, and delay spread parameters of the port operation area; A continuous communication quality distribution surface is generated based on the reference signal received power, signal-to-noise ratio, and delay spread parameters, and the continuous communication quality distribution surface is used as the communication probability field. Regions where the signal received power is lower than a first preset threshold or the multipath fading intensity is higher than a second preset threshold are marked as high-risk communication areas.

[0010] In an exemplary embodiment, the cross-modal feature fusion of image and audio information acquired by the UAV includes: Extract the audio feature vector of the audio information; Target detection is performed on image information to obtain the state features of moving targets within the line of sight; An attention mechanism is used to fuse the audio feature vector and the state feature to determine the geometric center coordinates and instantaneous velocity vector of the non-line-of-sight moving target.

[0011] According to another embodiment of the present invention, a port unmanned aerial vehicle (UAV) artificial intelligence scheduling system is provided, comprising: The information acquisition module is used to acquire port information and UAV information, wherein the port information includes port business characteristics, port dynamic elements, and port tasks, and the UAV information includes UAV flight performance. The flight area determination module is used to determine the flyable areas and no-fly zones based on the port information; The dynamic perception module is used to construct a time-series state vector based on the port's dynamic elements and to predict the time-series state vector in order to obtain dynamic obstacle distribution information within a preset number of steps in the future. The priority evaluation module is used to construct a task priority function based on the port business characteristics, and to determine the task priority of each port task according to the task priority function; The endurance verification module is used to determine the endurance verification result based on the flight performance of the UAV. The scheduling module is used to generate a multi-UAV scheduling sequence based on the task priority, endurance verification result, and flyable area, and to execute multi-UAV scheduling operations based on the multi-UAV scheduling sequence.

[0012] In one exemplary embodiment, the system further includes: The trajectory planning module is used to generate three-dimensional trajectory planning results based on preset constraints, and to perform real-time path replanning for the UAV based on the dynamic obstacle distribution information.

[0013] According to yet another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0014] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0015] This invention solves the obstacle avoidance problem caused by the highly dynamic port environment by making high-precision predictions of dynamic obstacles. At the same time, by setting priority functions, it achieves optimal resource allocation in complex business contexts, significantly enhancing the flight stability and operational safety of UAVs and greatly improving the collaborative efficiency of automated port operations. Therefore, it can solve the problem of low obstacle avoidance accuracy caused by the highly dynamic port environment and achieve the effect of improving the obstacle avoidance accuracy of port UAVs. Attached Figure Description

[0016] Figure 1 This is a flowchart of an artificial intelligence scheduling method for port drones according to an embodiment of the present invention; Figure 2 This is a structural block diagram of a port unmanned aerial vehicle (UAV) artificial intelligence scheduling system according to an embodiment of the present invention; Figure 3 This is a scheduling diagram according to Embodiment 2 of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0018] In the following description, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0019] Furthermore, in this application, directional terms such as "upper," "lower," "left," and "right" may be defined relative to the orientation of the components shown in the accompanying drawings. It should be understood that these directional terms can be relative concepts, used for relative description and clarification, and may change accordingly depending on the orientation of the components in the accompanying drawings.

[0020] In this application, unless otherwise expressly specified and limited, the term "connection" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral part; it can be a direct connection or an indirect connection through an intermediate medium. Furthermore, the term "coupled" can refer to an electrical connection that enables signal transmission.

[0021] As used herein, “about,” “approximately,” or “approximately” includes the stated value and the average value within an acceptable range of deviation from the given value, wherein the acceptable range of deviation is determined by a person skilled in the art taking into account the measurement under discussion and the error associated with the measurement of the given quantity (i.e., the limitations of the measurement system).

[0022] Example 1 This embodiment provides an artificial intelligence scheduling method for port unmanned aerial vehicles (UAVs). This method addresses the technical challenges faced by UAV operations in port environments, such as limited space, strong dynamic interference, and large micro-weather fluctuations, by constructing a real-time updated digital twin model, a precise task priority evaluation system, and a multi-scale flight trajectory planning algorithm. This application abstracts the port physical environment into a high-precision three-dimensional grid model and utilizes a deep learning model to predict the operating trajectories of mobile devices within the port, thereby providing a reliable spatiotemporal constraint basis for the collaborative scheduling of multiple UAVs.

[0023] In its implementation, this application constructs a highly real-time digital twin space by integrating port geographic information system (GIS) data, real-time berth layout, kinematic models of large cranes (such as quay cranes and yard cranes), and dynamic height data of containers in the yard. This space not only includes static buildings and terrain but also obtains information such as ship berthing plans and loading / unloading progress by connecting to the port management system, thereby achieving deep coupling between task priority and production rhythm at the scheduling level. To cope with adverse weather factors such as sea breezes and wind shear, this method also uses a multi-factor energy consumption model for endurance prediction and time-series analysis for flight safety control.

[0024] like Figure 1 As shown, the specific process includes the following: Step S100: Obtain port information and UAV information, and construct a three-dimensional spatial structure of the port based on the port geographic information system data, berth layout, crane model and yard height data contained in the port information. Then, generate a flyable area model using a three-dimensional rasterization method. The flyable area model includes a set of static obstacles, a set of moving obstacles dynamically updated based on a time-series prediction model, and flight corridors (i.e., flyable areas and no-fly zones) formed according to port safety regulations. The UAV information includes UAV flight performance.

[0025] In this embodiment, by fusing multi-source heterogeneous spatial data, the complex port operation area is transformed into a mathematical model that UAVs can understand; the spatial modeling module first extracts basic geographic information data of the port, including wharf shoreline, warehouse distribution and road grid.

[0026] Furthermore, by analyzing the berth layout and ship dynamics, the outline of the currently berthed ship can be determined; for the most disruptive equipment in the port—shore cranes (quay cranes) and rail-mounted gantry cranes—the space occupied by their beam extension and contraction, spreader lifting and lowering, and overall movement can be described based on their mechanical parameters and dynamic three-dimensional models fed back by real-time encoders.

[0027] Specifically, the 3D rasterization method maps the aforementioned spatial entities to a 3D spatial coordinate system, generating a 3D mesh model. In this model, a value of "1" indicates that the grid is occupied by a physical entity or is within a safe distance, and is marked as non-flying space; a value of "0" indicates flyable space; and the flyable region model... Defined as a set of triples .

[0028] For example, suppose the area of ​​a container terminal is [area range]. High coverage to The spatial modeling module divides the region into sections with sides of length [missing information]. Voxel unit; static obstacle set Includes registered fixed facilities such as command buildings, lighting poles, and fixed substations, whose position coordinates in the model remain constant over a long period; and a set of moving obstacles. It is a time-varying phenomenon. The changing function's core data comes from real-time feedback from the port equipment management system, such as the data of a machine numbered... The current pitch angle of the main beam of the quaybridge is... The lifting gear is located at a height At this point, the state is mapped in real time to The corresponding grid set; flight corridor These are safety passages pre-designed according to port regulatory procedures, such as those above crane operating areas. A buffer zone is established at each location; through this hierarchical description, the system can quickly retrieve any coordinate point. Is the passage condition met at the current moment?

[0029] Step S200: Obtain dynamic elements of the port through sensor components and construct a time series state vector. Use a long short-term memory network and a temporal convolutional network to predict the time series state vector to obtain dynamic obstacle distribution information within a preset number of steps in the future.

[0030] In this embodiment, the perception module integrates cameras, millimeter-wave radar, Automatic Identification System (AIS), and GPS of container trucks deployed throughout the port to acquire the real-time movement status of various operational elements. To proactively avoid future conflicts, a state vector containing multi-dimensional dynamic features can be constructed. ,in, This represents the average speed of the automated guided vehicle. The coordinates representing the key nodes of the crane. For real-time wind speed, This represents the current workload density, and so on.

[0031] This embodiment uses a Long Short-Term Memory (LSTM) network to capture the cyclical operation patterns of the container trucks in the yard, and a Temporal Convolutional Network (TCN) to collect position jumps caused by emergency braking of the crane; subsequently, the system obtains a prediction function by fusing the outputs of these two types of networks. Thus, the future can be calculated. Dynamic obstacle probability distribution at each time step .

[0032] For example, for a fleet of trucks in operation, sensors acquire their past... Trajectory data within, sampling frequency is State vector The dimension is set as Dimensions; LSTM layers contain Hidden layers, each with The TCN layer uses a residual structure and has 1 unit; the number of dilated convolutional layers is . Model outputs future A probability diagram showing the occupancy of each voxel; if at coordinate point... At that point, the prediction model is in The occupancy probability given at any given time exceeds the threshold. The scheduling system will detect flight conflicts in advance, thus preventing drones from entering the potentially dangerous area. This prediction-based perception mechanism, compared to traditional real-time feedback obstacle avoidance, can provide drones with a longer reaction time window and a smoother path selection scheme.

[0033] Step S300: Construct a task priority function based on the characteristics of port operations, and determine the task priority of each port task according to the task priority function. The relevant parameters of the task priority function include task urgency, security level, impact on port operations, time window constraints, and coverage requirements. The weights of the parameters are dynamically adjusted using a reinforcement learning algorithm.

[0034] In this embodiment, the port UAV missions cover various types, including ship berthing safety inspections, container overflow monitoring in the yard, quay crane structural fatigue detection, and emergency security response. The defined task priority function... Using linear weighting: .

[0035] in, For mission urgency (such as fire and smoke alarm missions) (far exceeding the level of routine inspections) Safety level, used to assess the risk level of the area where the mission is to be carried out; This represents the impact of the task on the port's core loading and unloading efficiency; Describe the time window constraint, that is, the task must be completed within a specific time period; The representative coverage requirement is used to assess the monitoring quality of key areas; while the weight vector is used to adapt to the needs of different port operation phases (such as busy and idle periods). The optimization goal is to maximize overall job efficiency and minimize task delays by using reinforcement learning algorithms for automated fine-tuning in the background.

[0036] For example, in a specific operational scenario, there are currently three concurrent tasks: Task A is a slope safety inspection of the newly berthed vessel "A" (with an extremely narrow time window); Task B is a periodic inspection of container tags in the yard; and Task C is perimeter protection of the port. At this time, the system's initial weight vector is... The reinforcement learning agent detects that the quay crane operation is at its peak based on the current real-time port throughput. At this point, task A... (Business impact) and The (time window) score improved significantly, and the calculated score was... , , Therefore, the scheduling system will prioritize assigning a UAV with the best imaging payload to Task A and plan the shortest path for it to ensure that safety confirmation is completed within the golden time before the ship loading and unloading begins.

[0037] Step S400: Using the UAV's flight performance as input, a multi-factor energy consumption model is used to predict the UAV's future flight power consumption, and mission feasibility analysis and return-to-home decision are made based on the prediction results.

[0038] The traditional constant power assumption fails in a port environment because the drag caused by sea winds drastically alters the energy consumption curve. In this embodiment, a multi-factor energy consumption model is used for prediction. Specifically, the multi-factor energy consumption model uses the UAV state vector... As input, where This represents the current battery percentage. It is the airspeed vector. The weight of the mounted equipment (such as infrared cameras, lidar, etc.). For ambient temperature, This refers to the attitude adjustment frequency caused by wind field disturbances.

[0039] The prediction formula is defined as follows , where parameters Trained using extensive historical flight data, the system reflects distance, wind resistance loss, and load loss; it continuously assesses whether the remaining battery power is sufficient to complete the remaining mission path and return to the charging station (reserved). (Safety margin).

[0040] For example, a machine with a weight , equipped A quadcopter drone carrying a payload is performing a mission at the port. Current battery level: Distance from charging station Real-time crosswind speed is The energy consumption prediction model, based on the current attitude stability coefficient (due to frequent high-power actions of the attitude motors caused by gusts), calculates that the energy consumption per unit distance is increased compared to a windless environment. The model calculation results show that after completing all observation points along the current route, the predicted power consumption upon returning to the base station is only [amount missing]. The value is below the safety threshold. The scheduling module immediately initiates intervention, reversing the original plan. The observation points were reduced to the highest priority. Alternatively, the drone can be instructed to switch to a low-energy cruising speed to ensure equipment safety.

[0041] Step S500: A time window constraint heuristic scheduling algorithm for port scenarios is adopted, which combines task priority, UAV endurance verification results, spatial reachability verification results, and local search optimization to generate a multi-UAV scheduling sequence.

[0042] In this embodiment, the improved time window heuristic algorithm TW-HS is used, and the scheduling objective function is set as follows: This is to minimize the weighted sum of total mission delay, overall flight risk, and total energy consumption.

[0043] During the search process, the algorithm eliminates candidate solutions that do not meet the power requirements through "endurance verification" and ensures that the planned path does not cross temporary no-fly zones caused by crane operations through "spatial accessibility verification". The local search optimization mechanism is used to find whether there is an improved solution that can further reduce the risk score after generating the initial sequence by swapping the execution order or execution machine of the two tasks.

[0044] For example, the existing port dispatch center Taiwanese drones and There are several tasks to be processed; the TW-HS algorithm first performs a preliminary sorting of the tasks based on the previously generated priority list. During the allocation process, the algorithm identifies tasks... Requirements to Complete the quay bridge between The hook inspection was being conducted, and at this time the drone... Located at Nearby. However, the battery life verification module reported... The battery was insufficient to support the subsequent high-altitude hovering maneuvers. The algorithm then searched for a suboptimal solution and found that the drone... Although the distance is slightly longer, it has a longer range, and by adjusting the inspection sequence in its path, it can... Reach the target location; the final generated scheduling sequence will include the tasks. Assigned to And at the global level, the on-time execution rate of all tasks reached [percentage missing]. And so on.

[0045] Step S600: Using the path search algorithm and minimum curvature optimization model, combined with port safety regulations, flight restricted areas and environmental interference constraints, a three-dimensional trajectory planning result is generated, and real-time path replanning is performed based on dynamic obstacle distribution information.

[0046] In this embodiment, it is also necessary to convert the macroscopic scheduling commands into smooth trajectories that the UAV can execute in order to achieve scheduling.

[0047] The system first uses A The path search algorithm finds an initial topological path within the previously generated 3D grid model, avoiding all known static and dynamic obstacles. Simultaneously, to satisfy the UAV's dynamic constraints and improve flight efficiency, the system then optimizes using a minimum curvature optimization model. This model aims to minimize the sum of squared curvatures of the trajectory, thereby reducing the UAV's deceleration and acceleration processes during turns. Its mathematical form is: [equation missing]. The minimum value of .

[0048] During execution, the system continuously receives dynamic obstacle prediction information. If it is found that the originally planned trajectory will overlap with a moving crane beam at some point in the future, the replanning module will immediately trigger a local correction.

[0049] For example, a drone is traversing a narrow corridor between two rows of container yards; A The algorithm generates an original path consisting of polylines, while the minimum curvature optimization module transforms these inflection points into continuous B-spline curves, enabling the UAV to... The drone maintained a constant speed and passed smoothly, avoiding significant tilting of the fuselage caused by sharp turns. When the drone reached the middle of the corridor, the perception system detected a truck illegally parked in the center. At this point, the replanning module responded quickly, recalculating the grid weights in a local area while keeping the global objective unchanged, generating a path to bypass the truck. The arc trajectory ensures that the position deviation calculation and command issuance delay are controlled within a certain range throughout the entire evasion process. This ensures the continuity of the flight.

[0050] In step S700, real-time control judgment is made based on the real-time monitored wind speed threshold, the status of sudden dynamic obstacles, and changes in the no-fly zone, and the threshold model is used to make time-series predictions of wind shear to trigger the flight stability protection strategy.

[0051] Because the port is located at the junction of land and sea, it is prone to localized strong winds or wind shear. The safety control module not only monitors real-time wind speed values, but also uses miniature pressure arrays and gyroscopes installed on the drone's sensors to build a wind field assessment model.

[0052] When the real-time wind speed detected exceeds the setting for this model Thresholds, or wind shear prediction algorithms, show future... When a significant change in lift occurs, the system will automatically trigger the flight stability protection strategy. This strategy is divided into three levels according to the risk level: Level 1 is to enhance attitude stability gain and reduce flight speed; Level 2 is to immediately abort the current mission and fly to the nearest safe landing point or shelter; Level 3 is to trigger the automatic return-to-home mechanism.

[0053] For example, during a nighttime inspection mission, the drone flew near the top of the quay crane. At this point, the safety control module, by analyzing the turbulence intensity of the airflow, predicted that there was severe downdraft (wind shear) at this location due to the influence of the crane's metal structure. The risk index calculated by the threshold model reached... Therefore, the system immediately took over control, abandoned the original low-altitude hovering maneuver, and instructed the drone to quickly climb to a safe altitude and switch to inertial navigation mode to resist magnetic field interference. At the same time, the background scheduling module received a safety warning, automatically marked the area as a "temporary high-risk zone," and notified subsequent inspection drones to detour. Through this real-time safety closed-loop control, the probability of drone crashes in the complex micro-meteorological environment of the port was significantly reduced.

[0054] This embodiment also provides a port UAV artificial intelligence scheduling system, which consists of an information acquisition module, a flight area determination module, a dynamic perception module, a priority evaluation module, an endurance verification module, a scheduling module, a trajectory planning module, and a safety control module. These modules are distributed on the cloud server and the edge of the UAV, and work collaboratively through a 5G high-reliability link. The spatial modeling unit in the information acquisition module uses geographic information system and crane encoder data to maintain a digital twin scene in real time. The dynamic perception module uses a long short-term memory network to predict obstacle trajectories. The priority evaluation module balances business needs through reinforcement learning. The endurance verification module ensures energy safety through a multi-factor model. The scheduling module is responsible for multi-UAV coordination, and the trajectory planning module is responsible for generating smooth paths. The safety control module handles sudden threats such as wind shear in real time.

[0055] The technical solution provided in this application achieves fully autonomous operation of drones in the extremely complex environment of ports through deep integration of artificial intelligence algorithms and port-specific business logic. Its advantages over existing technologies are reflected in the following aspects: First, by using digital twins and time-series prediction, obstacle avoidance capabilities are upgraded from "post-event response" to "pre-event prevention"; second, by using multi-factor energy consumption prediction and dynamic priority adjustment, the utilization rate of limited drone resources is improved; and third, by using minimum curvature optimization and safety threshold control, flight stability is ensured under strong winds and dynamic interference. This solution is not only applicable to container terminals but can also be extended to bulk cargo terminals, oil terminals, and similar complex industrial environments, possessing extremely high value for widespread application.

[0056] Example 2 To further enhance control precision, the following steps are also included: S110 constructs a port channel spatial fingerprint map based on port information, and superimposes a communication probability field on the three-dimensional spatial structure of the port by measuring the received power of the reference signal and the multipath fading parameters, so as to identify the signal shielding area and electromagnetic interference area generated by the metal container group.

[0057] In this embodiment, a three-dimensional communication probability field can be constructed to address the strong reflection and shielding effects of metal containers on electromagnetic waves. This field was obtained by combining road test data with the ray tracing algorithm.

[0058] The channel spatial fingerprint includes the reference received power (RSRP), signal-to-noise ratio (SINR), and root mean square delay spread (RMSDS); the signal attenuation model uses an improved empirical formula. in, This represents the additional shielding loss caused by the metal structure.

[0059] For example, such as Figure 3 As shown, in the "metal canyon" inside the storage yard, even when the drone is only a short distance from the ground base station... However, due to the obstruction of the container, the measured RSRP may be affected. sudden drop At this point, the system transforms discrete test points into a continuous probability field; if voxels Prediction and Then the voxel is assigned a high-risk weight. In subsequent path planning, the algorithm will tend to guide the drone above the top of the container. The aircraft flies within the strong signal area described above, rather than traversing gaps in the enclosure. For example, regarding coordinates... The deep mesh, surrounded by four layers of shipping containers, has a communication probability value set to... (Extremely low reliability) The dispatch system will strictly prohibit drones from entering the area without relay support, and so on.

[0060] The S210 utilizes the microphone array and visual sensor mounted on the UAV to perform cross-modal feature fusion, extracts the low-frequency acoustic signature of moving targets within the occluded area, and combines it with visual temporary information to achieve the localization and trajectory prediction of non-line-of-sight moving targets through a deep neural network.

[0061] In this embodiment, the audio-visual cross-modal fusion network includes an audio processing branch (based on a convolutional recurrent neural network (CRNN)) and a visual processing branch (based on a convolutional neural network (CNN)). The audio branch obtains the azimuth angle of the sound source by calculating the time difference of arrival (TDOA). The fusion layer employs a cross-attention mechanism to associate audio location information with historical trajectories in visual memory.

[0062] For example, the drone is flying north along aisle A of the storage yard, with a... The container truck is turning a corner. The vision sensors cannot see what's on the other side of the corner; however, the container truck emits an emission at a frequency of... The low-frequency noise characteristic of the area, which has diffraction capabilities, can bypass the corners of the container and be captured by the microphone array on the bottom of the drone. Therefore, the onboard four-microphone array captures a noise with an intensity of [insert value here]. The center frequency is The audio characteristics, after being processed by CRNN, were determined to be the sound of an approaching truck engine, with the sound source located slightly to the right of the front. Location, approximately Subsequently, the fusion network was combined with previous... Based on the observed traffic density in the area, it is predicted that the truck will... It then appeared at the corner. The system immediately sent a deceleration command to the drone and increased its flight altitude. Through this acoustic prediction, the drone completes the avoidance maneuver before visually detecting the target.

[0063] S310 uses a physical information-based neural network (PINNs) to perform real-time fluid dynamics simulation of the wake field generated by the crane beam and the ship's side, generating a three-dimensional flow field disturbance probability map, which is then used as an additional constraint for trajectory optimization.

[0064] To address airflow disturbances caused by the movement of large equipment, this embodiment utilizes deep learning through PINNs. Specifically, PINNs incorporates the Navier-Stokes (NS) equations as part of the loss function. This enables millisecond-level flow field prediction.

[0065] The model input is the descent speed of the quay crane spreader. Background wind speed The model output is the surrounding area. Three-dimensional vector field within the range .

[0066] For example, when a When a 10-foot container is rapidly lifted by a quay crane, a strong downdraft zone is formed beneath it, increasing wind speed. Reachable At this point, PINNs predicts the coordinates. There exists a Reynolds number in the region. Upon receiving information about turbulent air masses, the scheduling system adds an aerodynamic penalty term to the trajectory planning cost function. Therefore, the final planned trajectory will circle around the edge of this turbulent mass, maintaining at least... The lateral safety distance; of course, for a vehicle with a maximum wind resistance capacity of If the predicted composite wind speed of the flow field reaches the drone, The system will force a re-planning of the route.

[0067] S410 constructs a task priority function based on the characteristics of port operations, and determines the task priority of each port task according to the task priority function. The relevant parameters of the task priority function include task urgency, security level, impact on port operations, time window constraints, coverage requirements, and channel stability gain.

[0068] This embodiment adds a communication guarantee dimension to the original, and the priority function is updated as follows: in This represents the average signal reliability along the mission path. If a mission is urgent but must traverse a long signal dead zone, its priority weight will be appropriately reduced, or a "relay drone" collaborative mission will be triggered.

[0069] For example, a monitoring equipment malfunction alarm occurs at the port (Task A), although the urgency level is... High, but due to the fault point being located between two massive hulls, the signal coverage score was low. Only Subsequently, the reinforcement learning weight controller detected this risk and automatically triggered a collaborative solution: dispatching a high-performance, high-power drone as a communication relay at high altitude, while another inspection drone performed the task at a lower level. At this point, the overall priority of this task combination was increased. Prioritized over regular empty container inspection tasks.

[0070] The S510 takes the drone's flight performance as input and uses a multi-factor energy consumption model to predict the drone's future flight power consumption.

[0071] In this embodiment, based on the original multi-factor energy consumption model, the calculation of additional motor power consumption caused by frequent attitude adjustments in the strong turbulence region is added, for example, by adding an attitude correction power consumption term. .

[0072] S610 employs a time-window constraint-based heuristic scheduling algorithm for port scenarios to generate multi-UAV scheduling sequences that satisfy signal continuity constraints.

[0073] In one specific implementation, the TW-HS algorithm incorporates a hard constraint during the search process: the UAV must be in a low-signal area at any given time. The time shall not exceed This ensures that even in complex environments, the drone can periodically return to the strong signal area to report its status and calibrate the inertial navigation system.

[0074] The S710 uses a path search algorithm and a minimum curvature optimization model, combined with a three-dimensional flow field disturbance probability map, to generate trajectory planning results with aerodynamic stability, and performs real-time path replanning based on dynamic obstacle distribution information.

[0075] In a specific implementation, this step integrates the generated flow field information into A. Search, the cost of traversing a grid No longer just geometric distance, but .

[0076] For example, when planning a route from storage yard A to storage yard B, there are two possible routes: route one is shorter ( ), but it requires passing through a crane wake area and the signal is weak; path two is longer ( However, it is located in an open area with full signal strength; the optimization algorithm calculates the overall cost of path one as follows: Path 2 is Therefore, the system selected path two. During flight, if the perception fusion module detects a truck illegally entering a non-line-of-sight area, the system will... The new velocity vector is calculated based on the dynamic window method, and the centripetal force is adjusted to compensate for the sideslip caused by the airflow.

[0077] The S810 makes real-time control decisions based on real-time monitored wind speed thresholds, sudden dynamic obstacle status, prediction results of non-line-of-sight moving targets, and signal strength fluctuations.

[0078] In this embodiment, when Suddenly fell Upon arrival, it immediately enters the Visual Inertial Odometry (VIO) navigation mode and automatically climbs to find a signal.

[0079] This embodiment significantly enhances the innovation and practicality of the solution in real port environments by introducing communication probability fields, non-line-of-sight cross-modal sensing, and PINNs flow field simulation. The system can not only "see" obstacles but also "hear" dangers beyond its line of sight and "predict" invisible airflow interference and signal blind spots. This multi-dimensional deep integration solves the safety challenge of the last 100 meters in digital port operations, achieving truly intelligent scheduling across all port scenarios.

[0080] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0081] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0082] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0083] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0084] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0085] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0088] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0089] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0090] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0091] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A port drone artificial intelligence scheduling method, characterized in that, include: Acquire port information and drone information, wherein the port information includes port business characteristics, port dynamic elements, and port tasks, and the drone information includes drone flight performance. Based on the port information, the permitted and prohibited flight areas are determined. A time-series state vector is constructed based on the port's dynamic elements, and the time-series state vector is predicted to obtain dynamic obstacle distribution information within a preset number of steps in the future. Based on the port business characteristics, a task priority function is constructed, and the task priority of each port task is determined according to the task priority function; The endurance verification result is determined based on the flight performance of the UAV. Based on the task priority, endurance verification results, and flyable areas, a multi-UAV scheduling sequence is generated, and multi-UAV scheduling operations are executed based on the multi-UAV scheduling sequence.

2. The method according to claim 1, characterized in that, The method further includes: Based on preset constraints, a three-dimensional trajectory planning result is generated, and the UAV is replanned in real time according to the dynamic obstacle distribution information.

3. The method according to claim 1, characterized in that, The method further includes: Real-time control judgments are made based on real-time monitored wind speed thresholds, sudden dynamic obstacle status, and changes in no-fly zones to obtain real-time judgment results; as well as, Based on real-time monitored wind speed thresholds, time-series predictions of wind shear are performed to obtain flight stability protection strategies.

4. The method according to claim 1, characterized in that, After obtaining the port information, the method further includes: A communication probability field is constructed based on the port information, and the signal shielding zone and electromagnetic interference zone generated by the metal container group are determined through the communication probability field. Cross-modal feature fusion is performed on image and audio information collected by UAVs, and the location and trajectory prediction of non-line-of-sight moving targets are performed.

5. The method according to claim 4, characterized in that, The construction of the communication probability field based on the port information includes: Obtain the reference signal received power, signal-to-noise ratio, and delay spread parameters of the port operation area; A continuous communication quality distribution surface is generated based on the reference signal received power, signal-to-noise ratio, and delay spread parameters, and the continuous communication quality distribution surface is used as the communication probability field. Regions where the signal received power is lower than a first preset threshold or the multipath fading intensity is higher than a second preset threshold are marked as high-risk communication areas.

6. The method according to claim 4, characterized in that, The cross-modal feature fusion of image and audio information collected by the UAV includes: Extract the audio feature vector of the audio information; Target detection is performed on image information to obtain the state features of moving targets within the line of sight; An attention mechanism is used to fuse the audio feature vector and the state feature to determine the geometric center coordinates and instantaneous velocity vector of the non-line-of-sight moving target.

7. A port unmanned aerial vehicle (UAV) artificial intelligence scheduling system, characterized in that, include: The information acquisition module is used to acquire port information and UAV information, wherein the port information includes port business characteristics, port dynamic elements, and port tasks, and the UAV information includes UAV flight performance. The flight area determination module is used to determine the flyable areas and no-fly zones based on the port information; The dynamic perception module is used to construct a time-series state vector based on the port's dynamic elements and to predict the time-series state vector in order to obtain dynamic obstacle distribution information within a preset number of steps in the future. The priority evaluation module is used to construct a task priority function based on the port business characteristics, and to determine the task priority of each port task according to the task priority function; The endurance verification module is used to determine the endurance verification result based on the flight performance of the UAV. The scheduling module is used to generate a multi-UAV scheduling sequence based on the task priority, endurance verification result, and flyable area, and to execute multi-UAV scheduling operations based on the multi-UAV scheduling sequence.

8. The system according to claim 7, characterized in that, The system also includes: The trajectory planning module is used to generate three-dimensional trajectory planning results based on preset constraints, and to perform real-time path replanning for the UAV based on the dynamic obstacle distribution information.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is configured to perform the method described in any one of claims 1 to 6 when executed.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method as described in any one of claims 1 to 6.

Images