AI-driven 3D reconstruction method and system for UAV imagery in a distributed cluster environment

Abstract

This invention provides an AI-driven 3D reconstruction method and system for UAV imagery in a distributed cluster environment; it includes: acquiring image data from each node of the UAV cluster, performing feature extraction and 3D reconstruction on the image data; acquiring wind distortion parameters of the UAV flight area and computing power parameters of each node in real time, and establishing a coupling relationship between wind distortion and node computing power; correcting the feature extraction process of the image data based on the coupling relationship, simultaneously acquiring the time synchronization parameters of each node, and establishing a collaborative calculation relationship between the feature extraction results and the time synchronization parameters; optimizing the computing power allocation of the cluster nodes according to the collaborative calculation relationship, and then adjusting the wind distortion correction parameters and time synchronization parameters in reverse based on the optimized computing power allocation results. This invention can maintain high-speed solid-state drive writing under limited hardware interface resources to shorten the testing time.

Description

Technical Field

[0001] This invention relates to the field of technology, and in particular to an AI-driven 3D reconstruction method and system for UAV imagery in a distributed cluster environment. Background Technology

[0002] AI-driven 3D reconstruction technology based on UAV imagery in a distributed cluster environment has been widely applied in fields such as geographic surveying and mapping and emergency rescue scene reconstruction. Its core requirement is to collect images collaboratively by multiple UAV nodes and process them efficiently to output 3D models that meet the accuracy requirements of actual applications.

[0003] In existing technologies, optimizations for this type of 3D reconstruction often focus on improving a single aspect: either correcting image distortion caused by wind fields through image algorithms, alleviating insufficient computing power at nodes through simple computing power allocation, or reducing synchronization deviations between nodes through timestamp calibration. However, in practical applications, wind field distortion, node computing power attenuation, and time synchronization deviations have a significant coupled impact—strong wind fields exacerbate image feature distortion, leading to more computing power being required for feature extraction, while the dynamic attenuation of computing power at nodes due to battery consumption further reduces feature extraction efficiency; at the same time, feature processing delays caused by insufficient computing power amplify the existing time synchronization deviations between nodes, creating a vicious cycle of aggravated distortion, computing power strain, and synchronization inaccuracies, ultimately making it difficult for the 3D reconstruction accuracy to meet preset requirements, especially in scenarios with strong winds at high altitudes and a large number of cluster nodes, this problem is even more prominent.

[0004] Based on the above problems, there is an urgent need for a technical solution that can effectively resolve the coupling effects, break the vicious cycle, and ensure the accuracy of 3D reconstruction. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and propose an AI-driven 3D reconstruction method for UAV imagery in a distributed cluster environment, comprising:

[0006] Collect image data from each node of the drone swarm, extract features from the image data, and perform 3D reconstruction;

[0007] Real-time acquisition of wind field distortion parameters and computing power parameters of each node in the UAV flight area, and establishment of a coupling relationship between wind field distortion and node computing power;

[0008] Based on the feature extraction process of the coupled and correlated corrected image data, the time synchronization parameters of each node are obtained synchronously, and a collaborative calculation relationship between the feature extraction results and the time synchronization parameters is established.

[0009] The computing power allocation of cluster nodes is optimized based on the collaborative computing relationship. Then, the wind field distortion parameters and time synchronization parameters are adjusted in reverse based on the optimized computing power allocation results to form a closed-loop optimization of the three-dimensional reconstruction accuracy. Finally, a three-dimensional reconstruction model that meets the preset accuracy requirements is output.

[0010] Preferably, the real-time acquisition of wind field distortion parameters and computing power parameters of each node in the UAV flight area includes: collecting wind shear intensity and turbulence pulsation frequency of the wind field as wind field distortion parameters using a miniature ultrasonic anemometer; acquiring image distortion region masks using a fisheye camera and IMU to assist in correcting wind field distortion parameters; collecting the remaining power percentage of the node using an embedded power sensor; collecting the instantaneous CPU load rate of the node using a CPU load monitoring chip as computing power parameters; and predicting the computing power decay trend by combining the node's historical power consumption data. The computing power decay trend is used to predict the threshold for changes in feature extraction efficiency.

[0011] Further preferably, the feature extraction process based on the coupled and correlated corrected image data includes: when the instantaneous CPU load rate of the node showing the computing power attenuation trend exceeds a preset threshold, scheduling nodes in the cluster with instantaneous CPU load rates lower than the preset threshold to take over the image feature extraction task of that node; when the wind shear intensity exceeds a preset distortion threshold, enabling an optical flow-based image registration algorithm to correct the image distortion region, and then inputting the corrected image data into a preset AI feature extraction network for feature extraction, wherein the AI ​​feature extraction network outputs feature compensation coefficients to correct the feature extraction results.

[0012] A further preferred embodiment of the synchronous acquisition of time synchronization parameters for each node includes: collecting timestamp data of each node through a GPS module, collecting data transmission delay between each node through a millimeter-wave communication unit, combining the timestamp data and data transmission delay to form time synchronization parameters; and using a GPS and IMU dual-mode calibration algorithm to correct the time synchronization parameters and output a time synchronization correction coefficient, wherein the time synchronization correction coefficient is used to reduce the impact of data transmission delay on the synergy of feature extraction.

[0013] Further optimization involves establishing a coupling relationship between wind field distortion and node computing power, including calculating the effective image feature extraction rate using the following formula:

[0014] ;

[0015] In the formula: η F α represents the effective extraction rate of image features. W f is the wind shear intensity parameter. T β is the frequency parameter of turbulent fluctuations. B γ is the parameter representing the percentage of remaining power at a node. C ε is the instantaneous CPU load rate parameter for the node. FThe AI ​​feature compensation coefficient is output by a preset ResNet-50 network and has a value range of 0.05 to 0.2.

[0016] A further preferred approach is to establish a collaborative computation relationship between feature extraction results and time synchronization parameters, which includes calculating the collaborative computation efficiency of cluster nodes using the following formula:

[0017] ;

[0018] In the formula η C For the collaborative computing efficiency of cluster nodes, ΔT is the time difference parameter between nodes, and τ is the time difference parameter between nodes. D k is the data transmission delay parameter between nodes. S ω is the time synchronization correction coefficient. N δ is the parameter for the number of cluster nodes. C η is the computing power loss coefficient, ranging from 0.02 to 0.08. F The effective extraction rate of image features.

[0019] A further preferred method for achieving closed-loop optimization of 3D reconstruction accuracy includes calculating the 3D reconstruction spatial error using the following formula:

[0020] ;

[0021] In the formula σ R For the spatial error of 3D reconstruction, k R The accuracy coefficient is determined by the UAV image resolution and ranges from 0.05 to 0.2. σ0 is the basic error of 3D reconstruction and ranges from 0.02 to 0.05. ηC is the collaborative computing efficiency of the cluster nodes. When the spatial error of the 3D reconstruction exceeds the preset accuracy threshold, the weighted value of the remaining power ratio of the nodes and the wind field distortion correction parameters are adjusted, and the effective extraction rate of image features and the collaborative computing efficiency of the cluster nodes are recalculated until the spatial error of the 3D reconstruction is lower than the preset accuracy threshold.

[0022] Further preferred, optimizing the computing power allocation of cluster nodes includes: classifying computing power levels based on the collaborative computing efficiency of the cluster nodes, and dividing the computing power levels into high, medium, and low levels; dividing image feature extraction tasks into core tasks and auxiliary tasks according to complexity, allocating core tasks to nodes with high computing power levels, and allocating auxiliary tasks to nodes with medium and low computing power levels; monitoring changes in the computing power level of each node in real time, and when the computing power level of a node decreases, migrating the core tasks of that node to a node with a high computing power level to ensure continuous execution of feature extraction tasks.

[0023] An AI-driven 3D reconstruction system for UAV imagery in a distributed cluster environment is provided, applied to the AI-driven 3D reconstruction method for UAV imagery in a distributed cluster environment as described in any of the above-mentioned methods. The system comprises: a wind field-image distortion perception module, a node computing power dynamic monitoring module, a dual-mode time synchronization module, an AI feature extraction and collaborative computing module, a 3D reconstruction accuracy optimization module, and a main control module. The system is characterized in that the outputs of the wind field-image distortion perception module, the node computing power dynamic monitoring module, and the dual-mode time synchronization module are electrically connected to the input of the AI ​​feature extraction and collaborative computing module; the output of the AI ​​feature extraction and collaborative computing module is electrically connected to the input of the 3D reconstruction accuracy optimization module; the output of the 3D reconstruction accuracy optimization module is electrically connected to the input of the main control module; and the output of the main control module is electrically connected to the inputs of the wind field-image distortion perception module, the node computing power dynamic monitoring module, and the dual-mode time synchronization module. The main control module is used to coordinate the data interaction among the modules.

[0024] Further preferably, the wind field-image distortion perception module includes a miniature ultrasonic anemometer, a fisheye camera, and an IMU. The miniature ultrasonic anemometer is used to collect wind shear intensity and turbulent pulsation frequency, and the fisheye camera and IMU work together to obtain the image distortion region mask. The node computing power dynamic monitoring module includes an embedded power consumption sensor and a CPU load monitoring chip. The embedded power consumption sensor is used to collect the remaining power percentage of the node, and the CPU load monitoring chip is used to collect the instantaneous CPU load rate of the node. The dual-mode time synchronization module includes a GPS module and a millimeter-wave communication unit. The GPS module is used to collect timestamp data, and the millimeter-wave communication unit is used to collect data transmission delay. The AI ​​feature extraction and collaborative computing module includes an edge AI chip and a cluster communication bus. The edge AI chip adopts a Jetson Orin chip and stores the ResNet-50 network algorithm program. The cluster communication bus is used to transmit computing power scheduling instructions. The 3D reconstruction accuracy optimization module includes a cloud GPU cluster and a reconstruction engine. The cloud GPU cluster adopts an A100 GPU, and the reconstruction engine stores the 3D reconstruction error calculation program.

[0025] Technical Effects: This technology effectively solves the core problem in the background technology where wind field distortion, node computing power attenuation, and time synchronization deviation form a vicious cycle, leading to insufficient 3D reconstruction accuracy. It establishes a coupling relationship between wind field distortion and node computing power, and a collaborative calculation relationship between feature extraction results and time synchronization parameters, forming a closed-loop optimization for 3D reconstruction accuracy. This technique breaks the vicious cycle by dynamically adjusting wind field correction parameters, computing power allocation, and time synchronization parameters, ensuring that the output 3D reconstruction model meets the preset accuracy requirements, significantly improving the accuracy and stability of distributed cluster UAV 3D reconstruction. Attached Figure Description

[0026] Figure 1 This is a flowchart of the AI-driven 3D reconstruction method for UAV imagery in a distributed cluster environment, as described in this application.

[0027] Figure 2 This is a connection diagram of the AI-driven 3D reconstruction system for UAV imagery in a distributed cluster environment, as described in this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0029] Traditional distributed UAV 3D reconstruction technology ignores the coupled effects of wind field distortion, node computing power attenuation, and time synchronization deviation. Each processing module operates independently without a collaborative mechanism, resulting in low feature extraction rate, poor cluster processing efficiency, and ultimately, the 3D reconstruction accuracy cannot meet the preset requirements.

[0030] Based on this, please refer to Figure 1 This embodiment provides an AI-driven 3D reconstruction method for UAV imagery in a distributed cluster environment, including:

[0031] S1: Collect image data from each node of the drone swarm, extract features from the image data, and perform 3D reconstruction;

[0032] S2: Real-time acquisition of wind field distortion parameters and computing power parameters of each node in the UAV flight area, and establishment of a coupling relationship between wind field distortion and node computing power;

[0033] S3: Based on the feature extraction process of the coupled and correlated corrected image data, the time synchronization parameters of each node are obtained synchronously, and the collaborative calculation relationship between the feature extraction results and the time synchronization parameters is established.

[0034] S4: Optimize the computing power allocation of cluster nodes according to the collaborative computing relationship, and then adjust the wind field distortion parameters and time synchronization parameters in reverse based on the optimized computing power allocation results to form a closed-loop optimization of the three-dimensional reconstruction accuracy, and finally output a three-dimensional reconstruction model that meets the preset accuracy requirements.

[0035] It is worth mentioning that the above embodiments collect wind field distortion parameters and computing power parameters through corresponding sensing devices. The wind field distortion parameters include wind shear intensity, which reflects the severity of wind field changes, and turbulence pulsation frequency, which reflects the number of turbulence fluctuations per unit time. The computing power parameters include the remaining power percentage, which reflects the remaining power of the node, and the instantaneous load rate, which reflects the CPU's working intensity. Establishing a coupling correlation is to analyze the interaction between the increased computing power demand when wind field distortion intensifies and the further reduction in feature extraction capability due to computing power decay.

[0036] Based on this coupled and correlated feature extraction process, for example, when node computing power decays and causes an excessively high instantaneous load rate, low-load nodes within the cluster are scheduled to take over tasks. Simultaneously, time synchronization parameters, including inter-node time differences and data transmission delays, are collected via GPS and millimeter-wave communication units. Establishing a collaborative computing relationship correlates the completeness of feature extraction results with the accuracy of time synchronization parameters, determining their combined impact on cluster processing efficiency. Then, computing power allocation is optimized based on the collaborative computing results, such as assigning core extraction tasks to nodes with sufficient computing power and good time synchronization. The optimized computing power allocation results are then used to adjust wind field distortion parameters and time synchronization parameters in reverse, forming a closed loop of parameter acquisition, feature correction, collaborative computing, computing power optimization, and parameter adjustment. This loop iterates repeatedly until the 3D reconstruction accuracy meets the standards, ultimately outputting a 3D model that satisfies the preset requirements.

[0037] This solution addresses the coupling problem, achieves closed-loop optimization, and ensures that the output is a 3D reconstruction model that meets accuracy requirements.

[0038] Existing technologies lack image distortion masks to assist in correcting wind field parameters when acquiring wind field distortion and computing power parameters, and do not combine historical power consumption data to predict computing power decay trends, resulting in insufficient parameter accuracy and failing to provide a reliable basis for subsequent feature extraction and correction.

[0039] Based on this, the wind field distortion parameters and computing power parameters of each node in the UAV flight area are acquired in real time. This includes collecting wind shear intensity and turbulence pulsation frequency of the wind field as wind field distortion parameters using a miniature ultrasonic anemometer, and obtaining image distortion area masks using a fisheye camera and IMU to assist in correcting the wind field distortion parameters. The remaining power percentage of the nodes is collected by an embedded power sensor, and the instantaneous CPU load rate of the nodes is collected by a CPU load monitoring chip as computing power parameters. The computing power decay trend is predicted by combining the historical power consumption data of the nodes. The computing power decay trend is used to predict the threshold of change in feature extraction efficiency.

[0040] It is worth mentioning that in the above embodiments, the miniature ultrasonic anemometer calculates the wind shear intensity and turbulent pulsation frequency by emitting ultrasonic waves and detecting their propagation time difference. These two parameters are directly used as the initial wind field distortion parameters. The fisheye camera has a wide field of view and can capture complete images of the UAV's flight area. The IMU collects the UAV's attitude data in real time. By analyzing the correspondence between the pixel offset in the image and the IMU's attitude change, the image areas where texture stretching and offset are caused by the wind field are marked, generating a binary image distortion area mask. This mask is used to adjust the initial wind field distortion parameters. For example, in the high distortion areas marked by the mask, the value of the wind shear intensity is appropriately increased to improve the matching degree between the parameters and the actual distortion situation. The embedded power consumption sensor is connected in series in the node power circuit. By detecting the circuit current and voltage in real time and combining it with the total capacity of the node battery, the remaining power percentage is calculated. The CPU load monitoring chip obtains the ratio of the number of currently running tasks to the total number of CPU cores by reading the CPU's task scheduling register, thus obtaining the instantaneous load rate. These two parameters constitute the computing power parameters. Simultaneously, power consumption data of the node over a period of time is retrieved, the relationship between power consumption rate and load rate is analyzed, a fitting curve is established, and the future computing power decay trend is predicted. This trend can clearly identify the time point when the instantaneous load rate of the node reaches the threshold affecting the feature extraction efficiency, providing advance notice for subsequent task scheduling.

[0041] This solution improves the accuracy of wind field and computing power parameters, predicts computing power attenuation, and provides reliable data support for feature extraction and correction.

[0042] Existing technologies based on coupled correlation correction feature extraction do not have specific adjustment strategies for different scenarios such as excessive CPU load and strong wind shear, resulting in a lack of targeted correction and easy interruption or inaccurate results in feature extraction.

[0043] Based on this, the feature extraction process based on the coupled and correlated corrected image data includes: when the instantaneous CPU load rate of the node displaying the computing power attenuation trend exceeds a preset threshold, scheduling nodes in the cluster with instantaneous CPU load rates lower than the preset threshold to take over the image feature extraction task of that node; when the wind shear intensity exceeds a preset distortion threshold, enabling an optical flow-based image registration algorithm to correct the image distortion region, and then inputting the corrected image data into a preset AI feature extraction network for feature extraction, wherein the AI ​​feature extraction network outputs feature compensation coefficients to correct the feature extraction results.

[0044] It is worth mentioning that the CPU instantaneous load rate output for monitoring the computing power decay trend in the above embodiments is typically set to a preset threshold of 0.8. When the load rate of a node exceeds this threshold, the main control module retrieves the real-time load rate data of all nodes in the cluster through the cluster communication bus, filters out nodes with a load rate lower than 0.8, and prioritizes nodes with a remaining power percentage higher than 0.5. The image data of the nodes exceeding the threshold is migrated to the filtered nodes through a high-speed data link. During the migration process, the feature extraction task of the nodes exceeding the threshold is paused, and the task is restarted on the new node after the data transmission is completed, ensuring that the extraction process is not interrupted. When the wind shear intensity exceeds the preset distortion threshold, an optical flow-based image registration algorithm is first started. This algorithm calculates the motion vectors of corresponding pixels in two adjacent frames of images, establishes a spatial mapping relationship between the distorted area and the normal area, performs geometric transformation on the pixels in the distorted area, eliminates texture shift and stretching caused by the wind field, and obtains corrected image data. The corrected image data is input into a preset AI feature extraction network. This network extracts key features such as edges, corners, and textures from the image through 16 convolutional layers, 5 pooling layers, and fully connected layers. At the same time, the network outputs feature compensation coefficients based on the completeness of feature extraction. For areas with many missing features, the coefficients are used to weight and compensate for the surrounding complete features, correcting the extraction results and improving feature accuracy.

[0045] This solution provides precise correction for different scenarios that exceed the standard, ensuring continuous feature extraction and accurate results.

[0046] Existing technologies acquire time synchronization parameters by only collecting timestamps or transmission delays, without combining GPS and IMU dual-mode calibration to optimize parameters, and without utilizing the calibration results to reduce the impact of delay on feature extraction synergy, resulting in large synchronization parameter errors and asynchronous feature extraction among cluster nodes.

[0047] Based on this, the synchronous acquisition of time synchronization parameters of each node includes collecting timestamp data of each node through a GPS module, collecting data transmission delay between each node through a millimeter-wave communication unit, and combining the timestamp data and data transmission delay to form time synchronization parameters; the time synchronization parameters are corrected using a GPS and IMU dual-mode calibration algorithm, and a time synchronization correction coefficient is output. The time synchronization correction coefficient is used to reduce the impact of data transmission delay on the synergy of feature extraction.

[0048] It is worth mentioning that each cluster node in the above embodiments is equipped with a GPS module. This module receives signals from at least four satellites to generate timestamp data with millisecond-level accuracy. Each node uploads the timestamp data to the main control module. The millimeter-wave communication unit operates in the 24GHz or 77GHz frequency band. Each node sends test data packets to other nodes through this unit. The data packets contain the sending timestamp. The receiving node records the receiving timestamp and calculates the time difference between the two to obtain the data transmission delay between nodes. All delay data is summarized to the main control module. The time difference between nodes calculated from the timestamp data of each node is combined with the corresponding transmission delay to form a time synchronization parameter that includes time deviation and transmission time. A dual-mode calibration algorithm using GPS and IMU is used to correct this parameter. The GPS module provides a long-term stable time reference, but the signal is easily interrupted in obstructed scenarios. When the GPS signal is interrupted, the IMU calculates the node's motion state by detecting the node's angular velocity and acceleration, and performs short-term time compensation. The algorithm fuses the GPS and IMU data through Kalman filtering to correct the error in the time synchronization parameter and outputs the time synchronization correction coefficient. This correction coefficient is applied to collaborative control of feature extraction. For example, for node pairs with significant transmission delays, the feature data transmission time of the sending node is adjusted according to the correction coefficient, allowing the sending node to send data in advance. This ensures that the receiving node can start feature processing synchronously with other nodes, reducing the impact of delay on collaboration. This scheme improves the accuracy of time synchronization parameters, reduces the impact of delay, and enhances collaborative feature extraction.

[0049] Existing technologies do not quantify the coupling effect of wind field distortion and computing power on feature extraction. They only assess the extraction effect through qualitative judgment, which cannot accurately obtain the effective feature extraction rate. This results in a lack of quantitative basis for subsequent collaborative computing and computing power optimization.

[0050] Based on this, establishing the coupling relationship between wind field distortion and node computing power includes calculating the effective image feature extraction rate using the following formula:

[0051] ;

[0052] In the formula: η F α represents the effective extraction rate of image features. W f is the wind shear intensity parameter. T β is the frequency parameter of turbulent fluctuations. B γ is the parameter representing the percentage of remaining power at a node. C ε is the instantaneous CPU load rate parameter for the node. F The AI ​​feature compensation coefficient is output by a preset ResNet-50 network and has a value range of 0.05 to 0.2.

[0053] It is worth mentioning that the core of this formula is to quantify the impact of wind field distortion and computational power coupling on feature extraction, thereby obtaining the effective image feature extraction rate ηF, which is dimensionless and ranges from 0 to 1. The closer it is to 1, the more effective features there are. W The wind shear intensity parameter is dimensionless and ranges from 0.1 to 0.8. A larger value indicates more drastic wind field changes, more severe image distortion, and a greater negative impact on feature extraction. T This is the turbulence fluctuation frequency parameter, measured in Hz, with values ​​ranging from 1 to 10. Higher frequencies indicate more frequent changes between image frames per unit time, increasing the difficulty of feature matching; β B γ is a parameter representing the percentage of remaining power at a node. It is dimensionless and ranges from 0.2 to 1. A larger value indicates more abundant power at the node, a more stable computing power supply, and the ability to support more complete feature extraction. C ε is a node CPU instantaneous load rate parameter, dimensionless, ranging from 0.3 to 1. A larger value indicates higher CPU utilization, less idle computing power, and a decrease in feature extraction speed and completeness. F It is the AI ​​feature compensation coefficient, dimensionless, with a value of 0.05-0.2. It is output by the ResNet-50 network based on the completeness of the feature extraction results. When the proportion of missing features is high, the coefficient takes a larger value to compensate for extraction defects.

[0054] The calculation logic of this formula is divided into three parts. The first part is the negative impact term of the coupling between wind field and computing power, namely... α W The square is used to amplify the impact of wind shear intensity; the numerator reflects the combined effect of wind field distortion, and the denominator reflects the computing power's support capability. A larger ratio indicates a stronger negative impact. The second part is the positive impact of AI compensation, namely... In the exponential term, γ C The larger, α W The larger the value, the smaller the index, ensuring more accurate compensation when computing power is limited or wind conditions are severe. The third part is the final extraction rate calculation, which involves subtracting the negative impact term from 1 and adding the positive impact term to obtain η. F This value accurately reflects the effectiveness of feature extraction under current wind field and computing power conditions, providing a quantitative basis for subsequent collaborative computing. This scheme quantifies the coupling effect and accurately calculates the effective feature extraction rate, providing a reliable quantitative basis for subsequent collaborative processing.

[0055] Current technologies fail to quantify cluster collaborative computing efficiency by combining effective feature extraction rate and time synchronization parameters, relying solely on qualitative judgments to allocate computing power. This results in imprecise power allocation and poor overall cluster processing performance. Therefore, establishing a collaborative computing relationship between feature extraction results and time synchronization parameters involves calculating the collaborative computing efficiency of cluster nodes using the following formula:

[0056] ;

[0057] In the formula η C For the collaborative computing efficiency of cluster nodes, ΔT is the time difference parameter between nodes, and τ is the time difference parameter between nodes. D k is the data transmission delay parameter between nodes. S ω is the time synchronization correction coefficient. N δ is the parameter for the number of cluster nodes. C η is the computing power loss coefficient, ranging from 0.02 to 0.08. F The effective extraction rate of image features.

[0058] The purpose of the formula is to combine feature extraction results with time synchronization parameters to quantify the cluster collaborative computing efficiency η. C , dimensionless, ranging from 0 to 1, with values ​​closer to 1 indicating better collaboration. Here, ΔT is the time difference parameter between nodes, in milliseconds, ranging from 10 to 100, reflecting the deviation of timestamps between nodes; the larger the difference, the more difficult it is to synchronize the processing timing between nodes; τ D This is the data transmission latency parameter between nodes, measured in milliseconds (ms), and ranges from 5 to 50. A higher latency results in slower data interaction and lower collaborative processing efficiency; k S This is the time synchronization correction coefficient, dimensionless, ranging from 10 to 50, output by the GPS and IMU dual-mode calibration algorithm. A larger value indicates higher accuracy after time synchronization parameter correction, reducing the negative impact of time differences and delays; ω N This is a parameter representing the number of cluster nodes. It is dimensionless and ranges from 3 to 20. Too few nodes will lead to insufficient computing power, while too many will increase the complexity of data interaction. A moderate value should be chosen to optimize collaboration efficiency. δ C It is the computing power loss coefficient, which is dimensionless and ranges from 0.02 to 0.08. It reflects the loss caused by data transmission and task switching during the computing power allocation process. The smaller the value, the lower the loss.

[0059] The computational logic is divided into three parts. The first part is the basic collaborative efficiency term, namely η. F Multiply by time synchronization correction term The square of ΔT is used to amplify the effect of the time difference. This correction term reflects the adjustment effect of the time synchronization parameter on the coordination efficiency. ΔT and τ D The smaller, k S and ω N The more suitable the formula, the closer the correction term is to 1, and the closer the basic synergistic efficiency is to η. F The second part is the computing power loss item, namely... α reflects the degree of computing power strain. W The larger the value, the higher the computing power requirement due to wind field distortion. The exponential term is used to adjust the degree of loss, which needs to be deducted from the basic collaborative efficiency term. The third part is the final collaborative efficiency η. CThe value is obtained by subtracting the loss term from the basic term. This value can accurately reflect the collaborative processing capability of the cluster under the combined effects of feature extraction, time synchronization, and computing power loss, and provides a quantitative basis for subsequent computing power allocation optimization.

[0060] This scheme quantifies the efficiency of cluster collaborative computing, provides a precise basis for optimizing computing power allocation, and improves the cluster processing effect.

[0061] Existing technologies do not combine collaborative computing efficiency to quantify 3D reconstruction errors. They only adjust parameters through qualitative evaluation, which cannot form a closed-loop optimization, making it difficult to stably achieve the preset requirements for reconstruction accuracy.

[0062] Based on this, the closed-loop optimization of 3D reconstruction accuracy includes calculating the 3D reconstruction spatial error using the following formula:

[0063] In the formula: σ R For the spatial error of 3D reconstruction, k R η is the accuracy coefficient, determined by the UAV image resolution, ranging from 0.05 to 0.2; σ0 is the basic error of 3D reconstruction, ranging from 0.02 to 0.05; η is the accuracy coefficient, determined by the UAV image resolution, ranging from 0.05 to 0.2. C To improve the collaborative computing efficiency of cluster nodes, α W f T Δ T γ C β B The definitions of the corresponding parameters in the above embodiments are consistent; when the spatial error of the three-dimensional reconstruction exceeds the preset accuracy threshold, the weighted value of the remaining power ratio of the node and the wind field distortion correction parameter are adjusted, and the effective extraction rate of the image features and the collaborative computing efficiency of the cluster nodes are recalculated until the spatial error of the three-dimensional reconstruction is lower than the preset accuracy threshold.

[0064] The technical solution introduction needs to focus on error calculation and closed-loop optimization logic. The core of the formula is to quantify the spatial error σ of 3D reconstruction. R The unit is m, and the smaller the value, the higher the reconstruction accuracy. Where k... R σ0 is the accuracy coefficient, dimensionless, ranging from 0.05 to 0.2, determined by the UAV image resolution. Higher resolution results in a smaller coefficient, and lower resolution results in a larger coefficient, used to match the impact of different image qualities on the error; σ0 is the fundamental error of 3D reconstruction, measured in meters, ranging from 0.02 to 0.05, reflecting inherent errors caused by hardware and basic algorithms such as camera distortion and lens optical errors, which cannot be completely eliminated through parameter adjustment; η C α W f T , ΔT, γ C β B Consistent with the definitions in the above embodiments, ensuring parameter consistency.

[0065] The computational logic is divided into two parts. The first part is the error contribution term, namely... In molecules This reflects the contribution of wind field distortion to the error. The contribution of time synchronization and computing power coupling to the error is reflected in the denominator η. C β reflects the effect of collaborative efficiency on error suppression. B The first part reflects the effect of computing power stability on error suppression; the larger this contribution term, the more significant the error caused by external factors. The second part is the basic error correction term, i.e. The unit of ΔT needs to be converted to seconds (s). 0.01 is a constant to avoid the denominator being 0 when ΔT is 0. η C The larger the value of ΔT and the smaller the value of ΔT, the closer the exponent is to 1, and the closer the correction term is to σ0. Conversely, the smaller the value of ΔT and the smaller the value of ΔT, the closer the correction term is to σ0. This is used to adjust the impact of the basic error based on the coordination efficiency and time synchronization. The sum of the two parts is σ. R When σ R When the preset accuracy threshold is exceeded, a closed-loop adjustment is initiated: the weighting of the remaining power percentage of nodes is adjusted, assigning higher weights to nodes with a remaining power percentage higher than 0.7, prioritizing the allocation of core tasks to these nodes to improve computing power stability; wind field distortion correction parameters are adjusted, such as increasing the number of image registration iterations in high-distortion areas to enhance distortion correction effects; after adjustment, η is recalculated. F and η C , and then update σ R Repeat the iterations until σ R If the error falls below a threshold, a closed-loop optimization is initiated. This scheme quantifies the reconstruction error, achieves closed-loop optimization, and ensures that the accuracy of 3D reconstruction remains stable and meets the standards.

[0066] Existing technologies for optimizing cluster computing power allocation do not differentiate between computing power levels and task types, but simply assign tasks randomly, resulting in a waste of computing power resources. Furthermore, tasks are prone to interruption when node computing power decreases, affecting the continuity of feature extraction.

[0067] Based on this, optimizing the computing power allocation of cluster nodes includes classifying computing power levels based on the collaborative computing efficiency of the cluster nodes, dividing the computing power levels into high, medium, and low levels; dividing image feature extraction tasks into core tasks and auxiliary tasks according to complexity, allocating core tasks to nodes with high computing power levels, and allocating auxiliary tasks to nodes with medium and low computing power levels; and monitoring changes in the computing power level of each node in real time, and when the computing power level of a node decreases, migrating the core tasks of that node to a node with a high computing power level to ensure the continuous execution of feature extraction tasks.

[0068] The technical solution description needs to detail the level classification, task categorization, and migration logic. Firstly, based on the calculated cluster node collaborative computing efficiency ηC, the computing power levels are divided. Specific thresholds are set according to the actual application scenario, typically η... C≥0.8 indicates a high level of efficiency, with sufficient computing power and good time synchronization, enabling efficient processing of complex tasks; 0.5≤η C <0.8 indicates a medium level, with moderate collaborative efficiency, sufficient computing power, and the ability to handle tasks of moderate complexity. η C A value <0.5 indicates a low level, resulting in low collaborative efficiency, limited computing power, or poor time synchronization; it can only handle simple tasks.

[0069] The image feature extraction tasks are then categorized by complexity. Core tasks are operations that significantly impact the accuracy of 3D reconstruction, including feature extraction, feature matching, and fusion of keyframe images. Auxiliary tasks are operations that have a smaller impact on accuracy, including preprocessing of non-keyframe images and temporary storage and transmission of feature data.

[0070] During allocation, the main control module's computing power scheduling logic prioritizes core tasks for high-level nodes, with each high-level node receiving no more than 70% of its CPU cores as core tasks. Auxiliary tasks are allocated to mid- and low-level nodes, with mid-level nodes handling some simple auxiliary tasks and low-level nodes handling only basic auxiliary tasks such as data transmission. Simultaneously, a node computing power dynamic monitoring module collects the η (representational efficiency) of each node in real time. C Changes occur when the η of a certain high-level node changes. C When the performance drops below 0.8, the main control module immediately retrieves nodes that are still at a higher level and migrates any unfinished core tasks from those nodes to the new node via the cluster communication bus. Before migration, task progress data is saved, and execution resumes from the breakpoint on the new node after the transfer is complete, preventing task interruption. If the node level of an auxiliary task drops, it can decide whether to continue execution or migrate based on the task progress, ensuring the overall feature extraction task remains continuous and uninterrupted. This scheme standardizes computing power allocation, avoids resource waste, and ensures the continuous execution of feature extraction tasks.

[0071] The existing 3D reconstruction system has unclear module connections and lacks unified collaborative control among modules, making it impossible to achieve closed-loop execution of the method and thus preventing the system from being implemented.

[0072] Based on this, please refer to Figure 2This embodiment provides an AI-driven 3D reconstruction system for UAV imagery in a distributed cluster environment, including a wind field-image distortion perception module, a node computing power dynamic monitoring module, a dual-mode time synchronization module, an AI feature extraction and collaborative computing module, a 3D reconstruction accuracy optimization module, and a main control module. The output terminals of the wind field-image distortion perception module, the node computing power dynamic monitoring module, and the dual-mode time synchronization module are electrically connected to the input terminal of the AI ​​feature extraction and collaborative computing module, respectively. The output terminal of the AI ​​feature extraction and collaborative computing module is electrically connected to the input terminal of the 3D reconstruction accuracy optimization module, and the output terminal of the 3D reconstruction accuracy optimization module is electrically connected to the input terminal of the main control module. The output terminal of the main control module is electrically connected to the input terminals of the wind field-image distortion perception module, the node computing power dynamic monitoring module, and the dual-mode time synchronization module, respectively. The main control module is used to coordinate the data interaction of each module.

[0073] It is worth mentioning that this system comprises six core modules, each electrically connected to achieve data interaction and control command transmission. The output of the wind field-image distortion perception module is electrically connected to the input of the AI ​​feature extraction and collaborative computing module, transmitting data such as collected wind shear intensity, turbulence pulsation frequency, and image distortion mask to the AI ​​feature extraction and collaborative computing module, providing wind field data support for feature extraction correction. Similarly, the output of the node computing power dynamic monitoring module is electrically connected to the input of the AI ​​feature extraction and collaborative computing module, transmitting computing power parameters such as remaining power percentage, instantaneous CPU load rate, and computing power decay trend to support computing power scheduling for feature extraction. The output of the dual-mode time synchronization module is electrically connected to the input of the AI ​​feature extraction and collaborative computing module, transmitting synchronization parameters such as timestamp data, transmission delay, and time synchronization correction coefficient to support collaborative control of feature extraction.

[0074] After receiving data from the three perception modules, the AI ​​feature extraction and collaborative computing module performs feature extraction and calculates η. F and η C The operation involves electrically connecting the output to the input of the 3D reconstruction accuracy optimization module, which then processes the extracted feature data and η. C The data is then transmitted to the 3D reconstruction accuracy optimization module, providing the foundational data for error calculation. The 3D reconstruction accuracy optimization module calculates σ based on the received data. R It determines whether the preset threshold is exceeded, and its output terminal is electrically connected to the input terminal of the main control module to determine whether σ is exceeded. R Data and parameter adjustment instructions, such as instructions to adjust wind field correction parameters and computing power allocation weights, are transmitted to the main control module.

[0075] As the core of system collaboration, the main control module's output is electrically connected to the input of the wind field-image distortion sensing module, the node computing power dynamic monitoring module, and the dual-mode time synchronization module. Based on adjustment commands, it controls these modules to adjust their operating parameters, such as increasing the sampling frequency of the wind field-image distortion sensing module and strengthening the calibration frequency of the dual-mode time synchronization module. Simultaneously, the main control module coordinates the working sequence of the AI ​​feature extraction and collaborative computing module and the 3D reconstruction accuracy optimization module, ensuring smooth data transmission in the order of acquisition, processing, optimization, and adjustment. Ultimately, this achieves closed-loop control of the 3D reconstruction method, guaranteeing the successful implementation of system functions.

[0076] This solution clearly defines the module connection and collaboration logic, achieves closed-loop execution of methods, and ensures the implementation of system functions.

[0077] The hardware composition of each module in the existing 3D reconstruction system is unclear. The module functions are only mentioned without specifying the specific hardware, which makes it impossible to actually implement the module functions and makes the system difficult to deploy.

[0078] Based on this, the wind field-image distortion perception module includes a miniature ultrasonic anemometer, a fisheye camera, and an IMU. The miniature ultrasonic anemometer is used to collect wind shear intensity and turbulent pulsation frequency, while the fisheye camera and IMU work together to obtain image distortion region masks. The node computing power dynamic monitoring module includes an embedded power consumption sensor and a CPU load monitoring chip. The embedded power consumption sensor is used to collect the remaining power percentage of the node, while the CPU load monitoring chip is used to collect the instantaneous CPU load rate of the node. The dual-mode time synchronization module includes a GPS module and a millimeter-wave communication unit. The GPS module is used to collect timestamp data, and the millimeter-wave communication unit is used to collect data transmission delay. The AI ​​feature extraction and collaborative computing module includes an edge AI chip and a cluster communication bus. The edge AI chip uses a Jetson Orin chip and stores the ResNet-50 network algorithm program. The cluster communication bus is used to transmit computing power scheduling instructions. The 3D reconstruction accuracy optimization module includes a cloud GPU cluster and a reconstruction engine. The cloud GPU cluster uses an A100 GPU, and the reconstruction engine stores a 3D reconstruction error calculation program.

[0079] The technical solution description needs to clearly define the hardware composition and functional correspondence of each module. For the wind field-image distortion sensing module, a miniature ultrasonic anemometer with a range of 0-30 m / s is selected. It calculates wind shear intensity and turbulent pulsation frequency by using the time difference between transmitting and receiving ultrasonic waves. A fisheye camera with a resolution of 12 megapixels and a field of view of 190° is selected to acquire images of the UAV's surroundings. A six-axis IMU is selected to output angular velocity and acceleration data. Together, they generate an image distortion region mask using an image registration algorithm. For the node computing power dynamic monitoring module, an embedded power consumption sensor with a sampling rate of 1 kHz is selected and connected in series in the node power circuit. It calculates the remaining power percentage by detecting current and voltage. A CPU load monitoring chip supporting I2C communication is selected to obtain the instantaneous load rate by reading the CPU status register.

[0080] The GPS module of the dual-mode time synchronization module is a model that supports Beidou + GPS dual-mode with a time accuracy of 10ms, generating timestamp data; the millimeter-wave communication unit is a model that operates in the 24GHz band with a transmission rate of 100Mbps, and the transmission delay is calculated by sending test data packets.

[0081] The edge AI chip for the AI ​​feature extraction and collaborative computing module uses the Jetson OrinNX model, which has an AI computing power of 21 TOPS. It internally stores the ResNet-50 network algorithm program and can perform feature extraction and ηF and ηC calculations. The cluster communication bus uses the PCIe 4.0 bus with a transmission rate of 8GB / s and is used to transmit computing power scheduling instructions.

[0082] The cloud-based GPU cluster for the 3D reconstruction accuracy optimization module consists of four servers equipped with A100 GPUs. Each A100 GPU has 19.5 TFLOPS of double-precision computing power, enabling rapid calculation of σR. The reconstruction engine is a software module running on the GPU cluster, internally storing the 3D reconstruction error calculation program and capable of calling upon the GPU's computing power to execute formula calculations. All hardware selections consider industrial-grade stability and are suitable for the outdoor working environment of the drone swarm. This solution clearly defines the module hardware composition, ensuring the actual implementation of module functions and enabling system deployment.

[0083] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An AI-driven 3D reconstruction method for UAV imagery in a distributed cluster environment, characterized in that, include: Collect image data from each node of the drone swarm, extract features from the image data, and perform 3D reconstruction; Real-time acquisition of wind field distortion parameters and computing power parameters of each node in the UAV flight area, and establishment of a coupling relationship between wind field distortion and node computing power; Based on the feature extraction process of the coupled and correlated corrected image data, the time synchronization parameters of each node are obtained synchronously, and a collaborative calculation relationship between the feature extraction results and the time synchronization parameters is established. The computing power allocation of cluster nodes is optimized based on the collaborative computing relationship. Then, based on the optimized computing power allocation results, the wind field distortion parameters and time synchronization parameters are adjusted in reverse to form a closed-loop optimization of 3D reconstruction accuracy. Finally, a 3D reconstruction model that meets the preset accuracy requirements is output. Establishing the coupling relationship between wind field distortion and node computing power includes calculating the effective extraction rate of image features using the following formula: ； In the formula: η F α represents the effective extraction rate of image features. W f is the wind shear intensity parameter. T β is the frequency parameter of turbulent fluctuations. B γ is the parameter representing the percentage of remaining power at a node. C ε is the instantaneous CPU load rate parameter for the node. F The AI ​​feature compensation coefficient is output by a preset ResNet-50 network and ranges from 0.05 to 0.

2. Establishing the collaborative calculation relationship between feature extraction results and time synchronization parameters includes calculating the collaborative calculation efficiency of cluster nodes using the following formula: ； In the formula: η C For the collaborative computing efficiency of cluster nodes, ΔT is the time difference parameter between nodes, and τ is the time difference parameter between nodes. D k is the data transmission delay parameter between nodes. S ω is the time synchronization correction coefficient. N δ is the parameter for the number of cluster nodes. C η is the computing power loss coefficient, ranging from 0.02 to 0.

08. F The effective extraction rate of image features.

2. The AI-driven 3D reconstruction method for UAV imagery in a distributed cluster environment according to claim 1, characterized in that, Real-time acquisition of wind field distortion parameters and computing power parameters of each node in the UAV flight area includes: collecting wind shear intensity and turbulence pulsation frequency as wind field distortion parameters using a miniature ultrasonic anemometer; acquiring image distortion region masks using a fisheye camera and IMU to assist in correcting wind field distortion parameters; collecting the remaining power percentage of nodes using an embedded power sensor; collecting the instantaneous CPU load rate of nodes using a CPU load monitoring chip as computing power parameters; and predicting the computing power decay trend by combining historical power consumption data of nodes. The computing power decay trend is used to predict the threshold for changes in feature extraction efficiency.

3. The AI-driven 3D reconstruction method for UAV imagery in a distributed cluster environment according to claim 2, characterized in that, The feature extraction process based on the coupled and correlated corrected image data includes: when the instantaneous CPU load rate of the node showing the computing power decay trend exceeds a preset threshold, a node with a CPU instantaneous load rate lower than the preset threshold in the cluster is scheduled to take over the image feature extraction task of that node; when the wind shear intensity exceeds a preset distortion threshold, an image registration algorithm based on optical flow is enabled to correct the image distortion region, and then the corrected image data is input into a preset AI feature extraction network for feature extraction, and the AI ​​feature extraction network outputs feature compensation coefficients to correct the feature extraction results.

4. The AI-driven 3D reconstruction method for UAV imagery in a distributed cluster environment according to claim 1, characterized in that, The process of acquiring time synchronization parameters for each node includes: collecting timestamp data of each node through a GPS module, collecting data transmission delay between each node through a millimeter-wave communication unit, combining the timestamp data and data transmission delay to form time synchronization parameters; and using a GPS and IMU dual-mode calibration algorithm to correct the time synchronization parameters and output a time synchronization correction coefficient. The time synchronization correction coefficient is used to reduce the impact of data transmission delay on the synergy of feature extraction.

5. The AI-driven 3D reconstruction method for UAV imagery in a distributed cluster environment according to claim 1, characterized in that, The closed-loop optimization to achieve 3D reconstruction accuracy includes calculating the 3D reconstruction spatial error using the following formula: ； In the formula: σ R For the spatial error of 3D reconstruction, k R η is the accuracy coefficient, determined by the UAV image resolution, ranging from 0.05 to 0.2; σ0 is the basic error of 3D reconstruction, ranging from 0.02 to 0.05; η is the accuracy coefficient, determined by the UAV image resolution, ranging from 0.05 to 0.

2. C To improve the collaborative computing efficiency of cluster nodes; when the spatial error of the 3D reconstruction exceeds the preset accuracy threshold, the weighted value of the remaining power ratio of the nodes and the wind field distortion correction parameters are adjusted, and the effective extraction rate of image features and the collaborative computing efficiency of cluster nodes are recalculated until the spatial error of the 3D reconstruction is lower than the preset accuracy threshold.

6. The AI-driven 3D reconstruction method for UAV imagery in a distributed cluster environment according to claim 1, characterized in that, Optimizing the computing power allocation of cluster nodes includes: classifying computing power levels based on the collaborative computing efficiency of the cluster nodes, and dividing the computing power levels into high, medium, and low levels; dividing image feature extraction tasks into core tasks and auxiliary tasks according to complexity, allocating core tasks to nodes with high computing power levels, and allocating auxiliary tasks to nodes with medium and low computing power levels; monitoring changes in the computing power level of each node in real time, and migrating the core tasks of the node to the node with the current high computing power level when the node's computing power level decreases, to ensure the continuous execution of feature extraction tasks.

7. An AI-driven 3D reconstruction system for UAV imagery in a distributed cluster environment, applied to the AI-driven 3D reconstruction method for UAV imagery in a distributed cluster environment as described in any one of claims 1-6, characterized in that it comprises: The system comprises a wind field-image distortion perception module, a node computing power dynamic monitoring module, a dual-mode time synchronization module, an AI feature extraction and collaborative computing module, a 3D reconstruction accuracy optimization module, and a main control module. The outputs of the wind field-image distortion perception module, the node computing power dynamic monitoring module, and the dual-mode time synchronization module are electrically connected to the input of the AI ​​feature extraction and collaborative computing module. The output of the AI ​​feature extraction and collaborative computing module is electrically connected to the input of the 3D reconstruction accuracy optimization module. The output of the 3D reconstruction accuracy optimization module is electrically connected to the input of the main control module. The output of the main control module is electrically connected to the inputs of the wind field-image distortion perception module, the node computing power dynamic monitoring module, and the dual-mode time synchronization module. The main control module is used to coordinate the data interaction among the modules.

8. The AI-driven 3D reconstruction system for UAV imagery in a distributed cluster environment according to claim 7, characterized in that, The wind field-image distortion perception module includes a miniature ultrasonic anemometer, a fisheye camera, and an IMU. The miniature ultrasonic anemometer is used to collect wind shear intensity and turbulent pulsation frequency, while the fisheye camera and IMU work together to obtain image distortion region masks. The node computing power dynamic monitoring module includes an embedded power consumption sensor and a CPU load monitoring chip. The embedded power consumption sensor is used to collect the remaining power percentage of the node, and the CPU load monitoring chip is used to collect the instantaneous CPU load rate of the node. The dual-mode time synchronization module includes a GPS module and a millimeter-wave communication unit. The GPS module is used to collect timestamp data, and the millimeter-wave communication unit is used to collect data transmission latency. The AI ​​feature extraction and collaborative computing module includes an edge AI chip and a cluster communication bus. The edge AI chip uses a Jetson Orin chip and stores the ResNet-50 network algorithm program. The cluster communication bus is used to transmit computing power scheduling instructions. The 3D reconstruction accuracy optimization module includes a cloud GPU cluster and a reconstruction engine. The cloud GPU cluster uses an A100 GPU, and the reconstruction engine stores a 3D reconstruction error calculation program.

Images