A method, device and medium for unmanned aerial vehicle control for stockyard three-dimensional modeling

Abstract

The embodiment of the present application aims to provide a kind of unmanned aerial vehicle control method, equipment and medium for stockyard three-dimensional modeling, wherein the method comprises: generating multiple groups of acquisition points matched with target area according to the attributes of the target area to be modeled;Wherein, the number of acquisition points contained in each group of acquisition points is different;Multiple groups of acquisition points are iteratively optimized using genetic algorithm, and the group of acquisition points with the maximum fitness in the iteration optimization result is selected as the target acquisition point;Based on the target acquisition point, the optimal path is planned, and the target unmanned aerial vehicle is controlled to execute modeling task according to the optimal path.The present method iteratively analyzes the acquisition points for the target area, calculates the optimal number and position of acquisition points, and plans the optimal path for unmanned aerial vehicle flight modeling, greatly reduces the time required for blind repeated flight when unmanned aerial vehicle executes task, reduces the number of unmanned aerial vehicles required, reduces the probability of collision, and improves the execution efficiency of modeling task.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology, and in particular to a method, device and medium for controlling unmanned aerial vehicles (UAVs) for three-dimensional modeling of material yards. Background Technology

[0002] With the gradual saturation of the number of dry bulk cargo terminals along the coast and rivers in recent years, these terminals face a severe competitive environment characterized by homogeneity. Coupled with rising domestic labor costs and increasing awareness of human-centeredness and environmental protection, the extensive production methods and management mechanisms of the past, which relied solely on expanding investment to generate profits, are no longer suitable for the future development of bulk cargo terminals. At the same time, the emergence of technologies such as cloud services, mobile internet applications, sensors and the Internet of Things, big data, and artificial intelligence presents both challenges and new development opportunities for port management. Full automation of loading and unloading, and intelligent scheduling are gradually becoming rigid requirements for terminal development. This urgent need for automated loading and unloading has prompted existing automated dry bulk cargo terminals to research transformation and upgrading measures, aiming to build fully automated dry bulk cargo terminals that are safe, efficient, stable, reliable, and environmentally friendly.

[0003] Traditional dry bulk cargo terminals cannot accurately obtain information about material piles within the yard, nor can they acquire real-time data on the operational environment inside the ship's hold or the positioning information of unmanned mobile machinery within the hold. This prevents them from providing the foundational data for yard management and unmanned equipment control systems to support production organization decisions. Furthermore, they cannot achieve environmental perception of unmanned mobile machinery operating within semi-enclosed, high-dust ship holds, nor can they achieve real-time positioning and high-reliability data transmission for unmanned machinery. Therefore, to adapt to the trend of intelligent and automated terminals, existing technologies have led to solutions that manage the yard by establishing three-dimensional models.

[0004] In existing methods for modeling by acquiring image data from drones, in order to ensure the integrity of the established model, the drones need to perform reciprocating movements to ensure that the captured images can completely cover the area to be modeled. This not only requires the drones to work for a long time, but also requires a large number of drones. When multiple drones are operating, the probability of collision due to intersecting flight paths is relatively high. Summary of the Invention

[0005] The purpose of this invention is to provide a UAV control method, device, and medium for 3D modeling of material yards. The method in this embodiment calculates the optimal number and location of collection points by iteratively analyzing collection points in the target area, thereby planning the optimal path for UAV flight modeling. This greatly reduces the time required for UAVs to blindly and repeatedly fly during mission execution, reduces the number of UAVs required, lowers the probability of collisions, and improves the efficiency of modeling mission execution.

[0006] To address the above problems, embodiments of the present invention provide a UAV control method for 3D modeling of material yards, the method comprising:

[0007] Multiple sets of collection points matching the target region are generated based on the attributes of the target region to be modeled; each set of collection points contains a different number of collection points; a genetic algorithm is used to iteratively optimize the multiple sets of collection points, and the set of collection points with the highest fitness in the iterative optimization results is selected as the target collection point; an optimal path is planned based on the target collection point, and the target UAV is controlled to perform the modeling task according to the optimal path.

[0008] In addition, the step of generating multiple sets of collection points matching the target region based on the attributes of the target region to be modeled, as described above, includes: searching for an initial number of collection points matching the current attributes in a pre-built database based on the attributes of the target region to be modeled; selecting multiple prediction numbers based on the initial number of collection points; and generating a number of collection points equal to the prediction number within the target region as a group for each prediction number, thereby obtaining the multiple sets of collection points.

[0009] In addition, as described above, the genetic algorithm is used to iteratively optimize the multiple sets of collection points, including: conducting simulation experiments on each set of collection points to obtain multiple simulated point cloud data of the target area that correspond one-to-one with the multiple sets of collection points; calculating the fitness of each simulated point cloud data; performing cross-iteration on each set of collection points, conducting simulation experiments on each set of collection points after iteration, and calculating the corresponding fitness.

[0010] In addition, the fitness calculation of each simulated point cloud data as described above includes: obtaining the latest point cloud data of the target area; the latest point cloud data is the real point cloud data of the target area previously acquired by the UAV, or the reference point cloud data of the target area; calculating the similarity and F-score between each simulated point cloud data and the latest point cloud data respectively, and performing a weighted summation of the similarity and F-score values ​​to obtain the fitness of each simulated point cloud data.

[0011] Furthermore, the similarity as described above is calculated using the following formula:

[0012] Where P is the simulated point cloud data, and Q is the latest point cloud data; This indicates the similarity between P and Q. This represents the total number of points in P. This represents the total number of points in Q. Let i represent the i-th point in P. Let j be the j-th point in Q. express and The distance between them express and The distance between them For feature distance weight parameters, , express Local eigenvectors, express Local eigenvectors, express and The Euclidean distance between them express and The Euclidean distance between them.

[0013] In addition, as described above, the optimal path planning based on the target collection points includes: reading the position of each target collection point; and, based on the position of each target collection point and the pre-set positions of obstacles, starting point, and ending point, using a genetic algorithm to calculate the optimal path with the shortest path as the objective.

[0014] In addition, the method described above also includes: predicting the maximum power consumption of the drone executing the modeling task according to the optimal path; obtaining the current power of all drones currently in an idle state, and selecting any drone with a current power greater than the maximum power consumption as the target drone to execute the modeling task.

[0015] In addition, the method described above also includes: monitoring the battery level of the target drone during the modeling task; when the battery level of the target drone is lower than a preset threshold, controlling the target drone to return to the nearest landing point and recording the last target collection point passed by the target drone on the optimal path, using the target collection point as the starting collection point of the backup drone; controlling the backup drone to continue performing the modeling task from the starting collection point along the optimal path to the termination point.

[0016] This invention also proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the UAV control method for 3D modeling of a material yard as described above.

[0017] This invention also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the UAV control method for 3D modeling of a material yard as described above.

[0018] This invention's embodiments rationally determine multiple sets of collection points based on the attributes of the target area, and optimize the target collection points through a genetic algorithm. The optimized target collection points significantly improve coverage efficiency, allowing a single drone to complete the tasks that originally required multiple drones for zoned operations, greatly reducing the risk of collisions caused by the intersection of multiple drone routes during drone operations. Attached Figure Description

[0019] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0020] Figure 1 This is a flowchart illustrating the UAV control method for three-dimensional modeling of a material yard according to the first embodiment of the present invention.

[0021] Figure 2 This is a specific flowchart of step 101 in the second embodiment of the present invention;

[0022] Figure 3 This is a specific flowchart of step 102 in the third embodiment of the present invention;

[0023] Figure 4 This is a flowchart illustrating the calculation of fitness according to the third embodiment of the present invention;

[0024] Figure 5 This is a flowchart illustrating the UAV control method for 3D modeling of a material yard according to the fourth embodiment of the present invention.

[0025] Figure 6 This is a schematic diagram of the structure of the electronic device according to the sixth embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solutions of this disclosure, and to fully understand and implement the process of how this disclosure applies technical means to solve technical problems and achieve corresponding technical effects, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. The embodiments of this disclosure and the various features within them can be combined with each other without conflict, and the resulting technical solutions are all within the protection scope of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort should fall within the protection scope of this disclosure.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] The first embodiment of the present invention relates to a UAV control method for three-dimensional modeling of material yards, such as... Figure 1 As shown, the method in this embodiment specifically includes the following steps 101-103:

[0029] Step 101: Generate multiple sets of sampling points that match the target region based on the attributes of the target region to be modeled; the number of sampling points in each set of sampling points is different.

[0030] Specifically, users can select the material yard they want to model as needed, and then select a range within a preset distance centered on that material yard as the target area to be modeled. After selecting the target area, the attributes of the target area can be obtained through scanning or other methods. The attributes of the target area are characteristic features that may affect the number and distribution density of data collection points, etc., and this application does not limit the specific content of the attributes. In practical applications, the attributes of the target area may include: the area of ​​the target area, the complexity of the terrain (such as flat open space, bulk cargo stacks with varying elevations, areas with fixed equipment), the distribution of obstacles (such as the coordinates and height of cranes, the location and area occupied by container stacks), and the modeling accuracy requirements (such as 0.5m accuracy for material yard inventory statistics, and 0.2m accuracy for safety monitoring). These attributes are obtained through preprocessing after the target area is determined, such as obtaining an electronic map of the target area through the existing GIS system of the dock, and confirming the approximate height range of the material stacks through manual on-site surveys. This application also does not limit the specific method of determining the attributes.

[0031] After obtaining the target area to be modeled and its attributes, the distribution and number of data collection points are affected by the attributes of the target area. For example, the density of data collection points is positively correlated with the modeling accuracy. If the accuracy requirement is 0.5m, the spacing between data collection points needs to be controlled at 3-4m; if the accuracy requirement is 0.2m, the spacing between data collection points needs to be further reduced to capture more detailed material pile textures. Furthermore, the complexity of the terrain is positively correlated with the number of data collection points. In flat areas, data collection points can be generated according to the basic density; in areas with undulating material piles, additional data collection points need to be added at the top of the pile and on slopes; and the distribution of data collection points needs to be adjusted around obstacles. Therefore, this embodiment determines the data collection points for the target area based on its attributes. To ensure more accurate data collection point selection and guarantee subsequent modeling accuracy, this embodiment provides multiple sets of data collection points with different numbers based on the attributes of the target area to be modeled, providing more diverse candidate solutions for subsequent data collection point optimization.

[0032] Step 102: Use a genetic algorithm to iteratively optimize multiple sets of collection points, and select the set of collection points with the highest fitness in the iterative optimization results as the target collection points.

[0033] Specifically, after generating multiple sets of initial collection points, a genetic algorithm is used to iteratively optimize each set of collection points. The genetic algorithm includes a pre-defined fitness function to quantify the optimization results. The genetic algorithm also includes termination conditions for iterative optimization, which can be set and adjusted in practice. For example, a maximum number of iterations can be set, terminating the iteration when the maximum number of iterations is reached; or a minimum rate of change in fitness can be set, terminating the iteration when the rate of change in fitness before and after iteration is less than the minimum rate of change. The iteration process involves calculating the fitness of the newly generated set of collection points in each iteration. After the iteration is complete, the fitness of each set of collection points throughout the entire iteration process can be obtained. The set of collection points with the highest fitness is selected as the target collection point, which is the location that the UAV must visit during the modeling task of the target area.

[0034] Step 103: Plan the optimal path based on the target data collection points, and control the target UAV to perform the modeling task according to the optimal path.

[0035] Specifically, after determining the target data collection points, the optimal path for the UAV to perform the modeling task is planned. The core objective of the optimal path is to minimize the flight distance, the number of turns, and obstacle avoidance while ensuring that all target data collection points are visited, thereby further shortening the UAV's working time and reducing the risk of collision. This application does not impose specific restrictions on the algorithm for optimal path planning. For example, the A* algorithm can be used to find the shortest path, which is then selected as the optimal path. Alternatively, the A* algorithm combined with an obstacle avoidance algorithm can be used to plan the optimal path. In this case, the obstacle avoidance algorithm, based on the path planned by the A* algorithm, incorporates the previously acquired obstacle coordinates and height data, and then adjusts the path segment to avoid obstacles. For example, two temporary turning points can be set on both sides of the obstacle to allow the path to bypass the obstacle from the outside. Simultaneously, the UAV's flight altitude is adjusted according to the obstacle height. This application does not impose specific restrictions on the determination of the optimal path; the path that best meets the actual needs can be determined as the optimal path based on the actual target area conditions. The planned optimal path is converted into flight commands that the drone can recognize, including the precise coordinates of each data collection point. The flight commands are sent to the target drones through a dedicated drone communication system at the dock. After receiving the commands, the drones take off automatically and arrive at each target data collection point in sequence according to the optimal path. They hover, capture images, and store them. After all data collection points have been captured, the drones automatically return to the take-off and landing points, completing the mission.

[0036] Compared with related technologies, this embodiment rationally determines multiple sets of collection points based on the attributes of the target area, and optimizes the target collection points through a genetic algorithm. The coverage efficiency of the optimized target collection points is greatly improved, and a single drone can complete the task that originally required multiple drones to operate in different areas, greatly reducing the risk of collisions caused by the intersection of multiple drone routes during drone operations.

[0037] The second embodiment of the present invention relates to a UAV control method for three-dimensional modeling of a material yard. This embodiment is a supplement to the foregoing embodiment, and the supplement is that the step of generating multiple sets of acquisition points matching the target area in step 101 is refined, such as... Figure 2 As shown, the specific steps include the following.

[0038] Step 201: Based on the attributes of the target area to be modeled, find the number of initial collection points that match the current attributes in the pre-built database.

[0039] Specifically, this embodiment relies on a pre-built database. The data sources for this pre-built database can include historical operational data from 3D modeling of wharf material yards, industry standards and specifications, etc. This database establishes a mapping relationship between target area attribute combinations and the initial number of data collection points. Specifically, different levels can be assigned to each attribute, and attributes at each level can be combined with each other. For each combination, a historically verified initial number of data collection points is stored; this number represents an empirically reasonable value that meets modeling requirements, thereby establishing the mapping relationship.

[0040] For example, taking attributes including area, terrain complexity, and modeling accuracy as examples, area is divided into S1 (≤2500㎡), S2 (2500-10000㎡), and S3 (>100000㎡); terrain complexity is divided into T1 (flat without material stockpiles, such as an empty material yard), T2 (low-undulation material stockpiles, stockpile height ≤5m), and T3 (high-undulation material stockpiles, stockpile height >5m); modeling accuracy is divided into P1 (error tolerance 5%-8%), P2 (error tolerance 3%-5%), and P3 (error ≤3%). Attribute combination (S1+T1+P2): corresponds to 36 initial data collection points; attribute combination (S2+T3+P3): corresponds to 90 initial data collection points.

[0041] If no completely identical combination exists in the database, approximate matching is performed based on the priority between attributes. For example, the remaining items in the attributes are fixed first, and the combination with the priority precision closest to the highest priority attribute level is found. Its initial quantity is extracted, and then the number of collection points is adjusted according to the specific differences between attribute values ​​as the initial number of collection points for the current scene; or, the initial number of collection points is manually specified.

[0042] Step 202: Select multiple prediction quantities based on the initial number of collection points.

[0043] Specifically, the initial number of sampling points is determined, and multiple numbers are evenly selected around this initial number to form a differentiated gradient. The size of the gradient interval can be determined by the scale of the initial number, ensuring that the density difference of sampling points corresponding to each group can be identified and optimized by the genetic algorithm. The logic for setting the gradient interval is: the smaller the initial number, the smaller the interval ratio; the larger the initial number, the larger the interval ratio. This embodiment does not impose restrictions on the specific size of the interval or the number of predicted points, and can be adjusted according to the actual application. For example, when the initial number of sampling points is 20, 18, 19, 21, and 22 are selected before and after it with an interval of 1, along with the initial number of sampling points 20 itself, as multiple predicted points. When the initial number is 200, the gradient interval for selecting the predicted points can be set to ±5%, then the predicted points corresponding to 200 points are 180, 190, 200, 210, and 220. In addition, the number of sampling points can be adaptively adjusted according to the actual needs of modeling. This embodiment does not impose restrictions on the specific value of the predicted points; a predicted number greater than 2 is sufficient.

[0044] Step 203: Within the target area, generate a number of collection points equal to the predicted quantity for each predicted quantity, and form a group to obtain multiple groups of collection points.

[0045] Specifically, after obtaining multiple predicted quantities, a distribution of collection points is generated based on the shape of the target area to ensure uniform coverage of the collection points according to the predicted quantities. That is, each predicted quantity corresponds to a collection point distribution (a set of collection points). Collection points of the same number as each predicted quantity are randomly generated as a set of collection points, thus obtaining multiple sets of collection points. Based on the basic distribution of multiple sets of collection points, local optimization can be performed by combining terrain complexity and obstacle distribution to avoid collection points falling in invalid areas or missing key areas.

[0046] Compared with related technologies, this application determines the number of initial collection points based on a pre-built database, avoiding the problems of insufficient number leading to missed coverage or excessive number leading to increased energy consumption due to random generation. Each predicted number corresponds to a set of reasonably distributed collection points, ultimately forming multiple sets of collection points, providing a high-quality candidate set for subsequent genetic algorithm optimization.

[0047] The third embodiment of the present invention relates to a UAV control method for three-dimensional modeling of a material yard. This embodiment is a supplement to the foregoing embodiments, and the supplement lies in the refinement of step 102, such as... Figure 3 As shown, step 102 involves using a genetic algorithm to iteratively optimize multiple sets of data collection points, including:

[0048] Step 301: Conduct simulation experiments on each set of collection points to obtain multiple simulated point cloud data of the target area that correspond one-to-one with the multiple sets of collection points.

[0049] Specifically, after obtaining multiple sets of collection points in step 101, i.e., acquiring the location data of each set of collection points, a simulation experiment is conducted for each set of collection points. The simulation experiment is a virtual scenario that simulates the UAV performing modeling tasks according to the collection points, to ensure that the generated simulated point cloud data can accurately reflect the actual modeling effect. The specific implementation needs to be divided into three steps: simulation environment construction, collection process simulation, and point cloud data generation. Among them, the basic terrain model of the target area is imported into the simulation software, combined with obstacle 3D modeling, and UAV and camera parameter calibration to complete the simulation environment construction; the coordinate sequence of each set of collection points is imported into the simulation software, the flight path of the UAV is generated and the flight is simulated, and images are collected at each collection point location during the flight, and the images are distortion corrected; the simulated collected images are processed to generate simulated point cloud data corresponding to each set of collection points. That is, each set of collection points corresponds to one set of simulated point cloud data.

[0050] Step 302: Calculate the fitness of each simulated point cloud data.

[0051] Specifically, a multi-dimensional fitness calculation system can be designed based on the quality of the simulated point cloud. For example, the integrity, accuracy, uniformity, noise, etc. of the target area covered by each simulated point cloud data can be quantified, and weights can be assigned to the indicators of each dimension, and the fitness can be obtained by weighted summation. Alternatively, the fitness of the simulated point cloud data can be evaluated based on the correlation or similarity between the simulated point cloud data and the actual point cloud data.

[0052] In one example, such as Figure 4 As shown, step 302 specifically includes the following steps:

[0053] Step 401: Obtain the latest point cloud data for the target area; the latest point cloud data is the previously obtained real point cloud data for the target area, or the reference point cloud data for the target area.

[0054] Specifically, since modeling is a cyclical process, 3D modeling needs to be performed periodically to obtain the latest 3D model data. Therefore, for situations where the dockyard modeling frequency is high and the yard's condition changes relatively smoothly, the actual point cloud data of the target area from the previous modeling can be selected as the latest point cloud data. However, if the target area is being modeled for the first time, or if previous modeling data is lost or the yard's condition has changed drastically, reference point cloud data can be used as the latest point cloud data. The process of acquiring reference point cloud data can be as follows: a laser scanner is used to perform a full-coverage scan of the target area to generate a high-density reference point cloud; the precise coordinates of at least three reference points are obtained using GPS static measurement technology; the coordinates of the laser-scanned point cloud are calibrated; and areas irrelevant to the current modeling are cropped to obtain the reference point cloud data.

[0055] Point cloud data of the target area generated in the previous UAV operation is extracted from the modeling database, and the preprocessed point cloud data is converted into a coordinate system and data format consistent with the simulation point cloud. Alternatively, reference point cloud data is obtained through on-site scanning and converted into a coordinate system and data format consistent with the simulation point cloud.

[0056] Step 402: Calculate the similarity and F-score between each simulated point cloud data and the latest point cloud data respectively. Then, perform a weighted sum of the similarity and F-score values ​​to obtain the fitness of each simulated point cloud data.

[0057] Specifically, similarity is used to measure the consistency between the simulated point cloud and the benchmark data in terms of global shape and spatial distribution. The core is to evaluate the overall matching degree of the two sets of point clouds and avoid global offset or structural distortion in the simulation scheme.

[0058] The similarity in this embodiment is calculated using the following formula:

[0059] ;

[0060] Where P represents the simulated point cloud data, and Q represents the latest point cloud data. This indicates the similarity between P and Q. This represents the total number of points in P. This represents the total number of points in Q. Let i represent the i-th point in P. Let j be the j-th point in Q. express and The distance between them express and The distance between them For feature distance weight parameters, , express Local eigenvectors, express Local eigenvectors, express and The Euclidean distance between them express and The Euclidean distance between them;

[0061] in, , for The normal vector is obtained based on the PCA method. for The proportion (curvature) of the smallest eigenvalue of the covariance matrix. for The average distance (density) of k-nearest neighbors. k is The total number of points in the neighborhood of a given point. for The multi-scale features are the normal vectors, curvature, and density in different neighborhoods.

[0062] The F-score is a comprehensive metric that considers both recall and precision in point cloud calculations. It is used to evaluate the ability of a simulated point cloud to reproduce the local details of the latest point cloud, avoiding the loss or redundancy of local details in the simulated point cloud. The formula for calculating the F-score is as follows:

[0063] F-score=2*[(Pre1*Re1) / (Pre1+Re1)];

[0064] Wherein, Pre1 is the precision, calculated as the ratio of the number of valid matching points between the simulated point cloud and the latest point cloud to the total number of points in the simulated point cloud; Re1 is the recall, calculated as the ratio of the number of key points in the latest point cloud covered by the simulated point cloud to the total number of key points in the latest point cloud.

[0065] To comprehensively evaluate the global consistency and local detail matching of the simulated point cloud, the similarity and F-score values ​​are weighted and summed to obtain the final fitness. This embodiment does not restrict the specific weights of similarity and F-score values; they can be set according to the actual needs of modeling. For example, if the modeling focuses on the accuracy of the overall volume and spatial distribution of the material pile in the target area, with relatively low requirements for local detail accuracy, the similarity weight can be set to 0.6 and the F-score weight to 0.4 to ensure the dominant role of global matching in fitness. If the modeling focuses on local details such as cracks in the material pile and corners of obstacles in the target area, to avoid missing safety hazards due to lost details, the similarity weight can be set to 0.3 and the F-score weight to 0.7 to strengthen the dominant role of local detail matching in fitness. For example, if a dock needs to simultaneously consider inventory and safety requirements, the similarity and F-score weights can both be set to 0.5. After setting the weights, the fitness of each simulated point cloud is obtained by weighted summation.

[0066] Step 303: Perform cross-iteration on each group of data collection points, conduct simulation experiments on each group of data collection points after iteration, and calculate the corresponding fitness.

[0067] Specifically, each collection point is considered an individual in the genetic algorithm, and all collection points constitute the initial population (e.g., 5 collection points correspond to 5 individuals). Based on the total fitness score calculated in the first round, the top three individuals with the highest fitness are selected as the parent population. Crossover and mutation operations are performed on the parent population to obtain three new collection points after iterative optimization. Simulation experiments and fitness calculations are then performed on these three collection points. The parent and offspring populations are merged, sorted by total fitness score, and the top few are selected as the parent population for the next iteration. The above crossover-mutation-simulation-selection process is repeated until any of the following termination conditions are met:

[0068] 1. Iteration count met: The preset number of iterations has been reached;

[0069] 2. Fitness convergence: The improvement in the optimal fitness after N consecutive iterations is less than a preset threshold.

[0070] Compared with related technologies, the solution in this embodiment obtains a simulated point cloud by conducting simulation experiments on each set of collection points and then calculates the fitness. This eliminates the need to verify the collection point scheme through actual drone flights, thus avoiding drone collisions caused by unreasonable collection points. By dynamically selecting the latest point cloud for comparison with the simulated point cloud, the problem of the simulation model being out of sync with the real material yard is solved, making the fitness assessment more accurate and in line with the actual situation.

[0071] The fourth embodiment of the present invention relates to a UAV control method for 3D modeling of a material yard. This embodiment is a supplement to the foregoing embodiments, and the supplement is that the content of planning the optimal path in step 103 is refined, including:

[0072] The location of each target acquisition point is read; based on the location of each target acquisition point and the pre-set locations of obstacles, starting point and ending point, the optimal path is calculated using a genetic algorithm with the goal of finding the shortest path.

[0073] Specifically, in this embodiment, the optimal path refers to the length of the path, and the shortest path is taken as the optimal path. In practical applications, the optimal path can be set according to actual needs, such as planning that the drone must pass through certain points in sequence. This embodiment does not impose specific restrictions on the actual content of the optimal path.

[0074] When using the shortest path as the optimal path, the three-dimensional coordinates of each point are first extracted from the established set of target data collection points. Three-dimensional boundary data of obstacles are imported from the dock map system, and the start and end points of the UAV flight are set, ensuring all positions are aligned to the same coordinate system. Then, with the goal of minimizing the total path length while avoiding obstacle crossings, a genetic algorithm is used for path optimization. The optimized path sequence is converted into a sequence of waypoint coordinates executable by the UAV, including parameters such as the position, flight speed, and hovering time of each point, thus obtaining the optimal path.

[0075] This embodiment uses a genetic algorithm combined with obstacle constraints and the shortest path objective to plan the optimal path. Compared to traditional reciprocating UAV flights, the flight path is significantly shortened, and collisions with fixed obstacles at the dock are avoided, thus improving flight safety.

[0076] The fifth embodiment of the present invention relates to a UAV control method for three-dimensional modeling of a material yard. This embodiment is a supplement to the foregoing embodiments, and the supplement is made to the process of determining the target UAV, specifically including:

[0077] Predict the maximum power consumption of the drone executing the modeling task according to the optimal path;

[0078] Obtain the current battery level of all drones that are currently idle, and select any drone with a current battery level greater than the maximum power consumption as the target drone to perform the modeling task.

[0079] Specifically, based on the optimal path parameters and the energy consumption ratio of drone flights obtained from historical experience, the maximum power consumption required to complete the task is calculated, ensuring that the prediction results have a margin of safety. The drone management system collects the status information of all drones in real time, filters out idle drones (not performing tasks and without faults), and obtains their current power information. From the idle drones, drones with current power greater than the maximum power consumption are selected. If multiple drones meet the criteria, selection can be based on priority conditions, such as: prioritizing the drone with the closest power consumption to the maximum, prioritizing the drone with the longest rest time, or prioritizing the drone closest to the start / end point. If the power of all idle drones does not meet the requirements, the system will issue an alarm, prompting manual battery replacement or task delay.

[0080] In another example, such as Figure 5 As shown, considering the potential for insufficient battery power due to external environmental interference such as wind resistance or sudden malfunctions during drone missions, this embodiment provides the following method to ensure continuous execution of the modeling task:

[0081] Step 501: Monitor the battery level of the target drone during the modeling task.

[0082] Specifically, when the target drone is performing a mission, it can collect power data in real time through the battery management system and send the current power level and flight distance back to the control center every few seconds.

[0083] Step 502: When the target drone's battery level is below a preset threshold, control the target drone to return to the nearest landing point and record the last target collection point that the target drone passes through on the optimal path, and use this target collection point as the starting collection point for the backup drone.

[0084] Specifically, a preset threshold is set for the battery monitoring alarm, such as 20%. When the drone's battery level is detected to be below the threshold, a return command is triggered, controlling the drone to interrupt the current mission and return to the nearest landing point. The drone's location is used to record the last target acquisition point that has been captured (e.g., the 36th acquisition point), storing its coordinates, shooting time, image number, and other information as the starting point for subsequent relays. During the return process, the target drone prioritizes transmitting the captured image data to avoid data loss.

[0085] It should be noted that the selection of a backup drone can be synchronized with the selection of the target drone. For example, when selecting a target drone, a drone with a current battery level greater than its maximum power consumption can be chosen from among the idle drones. If multiple drones meet the criteria, the one with the highest battery level can be selected as the target drone, and the one with the second highest battery level can be selected as the backup drone. Alternatively, the power consumption of the remaining path can be evaluated, and a drone with a battery level greater than the power consumption of the remaining path can be selected as the backup drone from among the idle drones. This embodiment does not restrict the method for determining the backup drone, as long as its remaining battery level is sufficient to support it in completing the remaining optimal path.

[0086] Step 503: Control the backup drone to continue the modeling task from the starting point to the ending point along the optimal path.

[0087] Specifically, a backup drone is automatically activated to take over, sending the optimal path, breakpoint information (the 36th data collection point), and a list of completed data collection points to the backup drone, ensuring that the backup drone knows the remaining path to be executed (from the 37th to the 60th data collection point). The backup drone is controlled to fly over the breakpoint data collection point (the 36th point) and hover, calibrating its position through visual positioning (identifying ground markers) to ensure continuity with the original path; it then starts taking pictures from the 37th data collection point along the optimal path, maintaining flight parameters (speed, altitude) consistent with the original drone to ensure consistency during image stitching; after the task is completed, it returns and merges the newly acquired images with the original data to form a complete modeling dataset.

[0088] This embodiment ensures uninterrupted modeling tasks through low battery monitoring and backup drone takeover, solving the problem of data loss caused by sudden power shortages in traditional single-drone operations. In practical applications, even if the original drone returns midway, the backup drone can still guarantee the integrity of the modeling data.

[0089] The sixth embodiment of the present invention relates to an electronic device, such as... Figure 6 As shown, it includes at least one processor 602; and a memory 601 communicatively connected to at least one processor 602; wherein the memory 601 stores instructions executable by at least one processor 602, the instructions being executed by at least one processor 602 to enable at least one processor 602 to perform any of the above method embodiments.

[0090] The memory 601 and processor 602 are connected via a bus, which may include any number of interconnecting buses and bridges. The bus connects various circuits of one or more processors 602 and memory 601 together. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 602 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 602.

[0091] Processor 602 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 601 can be used to store data used by processor 602 during operation.

[0092] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements any of the above-described method embodiments.

[0093] Another embodiment of the present invention relates to a computer program product, including computer instructions that, when executed by a processor, implement any of the above-described method embodiments.

[0094] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0095] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0096] While the embodiments disclosed herein are as described above, the foregoing content is merely for the purpose of facilitating understanding of this disclosure and is not intended to limit this disclosure. Any person skilled in the art to which this disclosure pertains may make any modifications and changes in form and detail of the implementation without departing from the spirit and scope of this disclosure; however, the scope of patent protection of this disclosure shall still be determined by the scope defined in the appended claims.

Claims

1. A UAV control method for 3D modeling of material yards, characterized in that, include: Multiple sets of sampling points matching the target region are generated based on the attributes of the target region to be modeled; wherein the number of sampling points in each set of sampling points is different; A genetic algorithm is used to iteratively optimize the multiple sets of collection points, and the set of collection points with the highest fitness in the iterative optimization results is selected as the target collection point. Based on the target data collection points, plan the optimal path and control the target UAV to perform the modeling task according to the optimal path; The step of generating multiple sets of sampling points matching the target region based on the attributes of the target region to be modeled includes: Based on the attributes of the target area to be modeled, find the number of initial collection points that match the current attributes in a pre-built database; Multiple prediction quantities are selected based on the initial number of collection points; Within the target area, based on each predicted quantity, a number of collection points equal to the predicted quantity are generated as a group, resulting in the multiple groups of collection points; The step of using a genetic algorithm to iteratively optimize the multiple sets of collection points includes: Simulation experiments were conducted on each set of collection points to obtain multiple simulated point cloud data of the target area that correspond one-to-one with the multiple sets of collection points; Calculate the fitness of each of the simulated point cloud data; Cross-iteration is performed on each group of data collection points, and simulation experiments are conducted on each group of data collection points after iteration, and the corresponding fitness is calculated.

2. The method according to claim 1, characterized in that, The calculation of the fitness of each of the simulated point cloud data includes: Obtain the latest point cloud data of the target area; the latest point cloud data is the real point cloud data of the target area previously acquired by the UAV, or the reference point cloud data of the target area; The similarity and F-score between each of the simulated point cloud data and the latest point cloud data are calculated respectively. The similarity and F-score are then weighted and summed to obtain the fitness of each of the simulated point cloud data.

3. The method according to claim 2, characterized in that, The similarity is calculated using the following formula: ； Where P represents the simulated point cloud data, and Q represents the latest point cloud data. This indicates the similarity between P and Q. This represents the total number of points in P. This represents the total number of points in Q. Let i represent the i-th point in P. Let j be the j-th point in Q. express and The distance between them express and The distance between them For feature distance weight parameters, , express Local feature vectors, express Local feature vectors, express and The Euclidean distance between them express and The Euclidean distance between them.

4. The method according to claim 1, characterized in that, The step of planning the optimal path based on the target collection point includes: Read the location of each target acquisition point; Based on the location of each target acquisition point and the pre-set locations of obstacles, starting point, and ending point, the optimal path is calculated using a genetic algorithm with the goal of minimizing the path length.

5. The method according to claim 1, characterized in that, The method further includes: Predict the maximum power consumption of the drone executing the modeling task according to the optimal path; Obtain the current battery level of all drones currently in an idle state, and select any drone whose current battery level is greater than the maximum power consumption as the target drone to perform the modeling task.

6. The method according to claim 1, characterized in that, The method further includes: During the modeling task performed by the target drone, the battery level of the target drone is monitored; When the target drone's battery level is below a preset threshold, the target drone is controlled to return to the nearest landing point, and the last target collection point passed by the target drone on the optimal path is recorded, and this target collection point is used as the starting collection point for the backup drone. The backup drone is controlled to continue performing the modeling task from the starting point to the ending point along the optimal path.

7. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that are executed by the at least one processor to enable the at least one processor to perform the UAV control method for three-dimensional modeling of a material yard as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the UAV control method for three-dimensional modeling of material yards as described in any one of claims 1 to 6.

Images