Multi-uav cooperation-based moving object three-dimensional model reconstruction system and method

Abstract

The application belongs to the technical field of model three-dimensional reconstruction, and particularly relates to a moving object three-dimensional model reconstruction system and method based on multi-unmanned aerial vehicle cooperation. The system is composed of multiple unmanned aerial vehicles. Each unmanned aerial vehicle is loaded with a self-organizing network communication module, a data acquisition module, a time marking module, a state prediction module, a trajectory prediction module and a flight control module. Through adaptive cooperative observation of multiple unmanned aerial vehicles, adaptive view angle optimization driven by trajectory prediction, space-time alignment and multi-level image fusion, high-precision and real-time three-dimensional reconstruction of a moving object is realized.

Description

Technical Field

[0001] This invention belongs to the field of 3D model reconstruction technology, specifically relating to a 3D model reconstruction system and method for moving objects based on multi-UAV collaborative operation. Background Technology

[0002] With the rapid development of intelligent transportation and autonomous driving, accurately acquiring 3D models of moving objects (such as vehicles, driverless vehicles, and pedestrian groups) has become a crucial aspect of perception and monitoring. Existing technologies primarily employ the following methods:

[0003] Footage captured by a single drone. A single drone tracks a moving object and captures a video stream, then stitches the images together to achieve 3D reconstruction. However, because drones struggle to maintain optimal angles, and moving objects move at high speeds and change directions frequently, image distortion and data loss are common problems.

[0004] Multi-view stitching. This involves stitching together images taken at different times and from multiple angles to reconstruct the image. However, in practical applications, due to factors such as lighting, environmental noise, and changes in the state of moving objects, there is often a discrepancy between the time scale and the spatial scale, making it difficult to guarantee stitching accuracy.

[0005] Fixed surveillance cameras combined with algorithms. Although they can cover a partial area, the fixed camera positions prevent flexible adjustment of the viewing angle, resulting in poor reconstruction results for high-speed or irregularly moving objects.

[0006] Therefore, how to achieve three-dimensional reconstruction of moving objects in a dynamic environment by leveraging the collaborative advantages of multiple drones while maintaining real-time performance and accuracy is a technical problem that urgently needs to be solved. Summary of the Invention

[0007] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0008] Firstly, a system for reconstructing 3D models of moving objects based on multi-UAV collaborative operation is provided, comprising multiple UAVs; each UAV is equipped with:

[0009] The self-organizing network communication module is used to establish a two-way communication link between this drone and each of the other drones to form a mesh network;

[0010] The data acquisition module is used to synchronously acquire the flight status parameters of this UAV and the motion images of the target object, add timestamps to the flight status parameters and motion images, and call the self-organizing network communication module to broadcast the flight status parameters, motion images and corresponding timestamps through the mesh network;

[0011] The state prediction module is used to predict the motion state of the target object at the next data acquisition time based on the flight state parameters of the UAV at the current data acquisition time and the motion image of the target object at the current data acquisition time, so as to obtain the global state vector set of the target object at the next data acquisition time.

[0012] The point acquisition module is used to obtain the best observation point of the UAV at the next data acquisition time based on the global state vector set, shooting angle constraints, relative distance constraints between the UAV and the target object, and the position of the UAV at the current data acquisition time.

[0013] The flight control module is used to control the UAV to fly from its current location at the time of data acquisition to the target observation point at the next time of data acquisition.

[0014] The 3D reconstruction module is used to reconstruct the 3D model of the target object using motion images collected by this UAV at the corresponding target observation point and motion images collected by other UAVs at the corresponding target observation point.

[0015] Secondly, a method for reconstructing a 3D model of a moving object based on multi-UAV collaboration is provided, which involves performing the following steps on each UAV:

[0016] Establish a two-way communication link between this drone and every other drone to form a mesh network;

[0017] The system synchronously collects the flight status parameters of the UAV and the motion images of the target object, adds timestamps to the flight status parameters and motion images, and calls the self-organizing network communication module to broadcast the flight status parameters, motion images and corresponding timestamps through the mesh network.

[0018] Based on the flight state parameters of the UAV at the current data acquisition time and the motion image of the target object at the current data acquisition time, the motion state of the target object at the next data acquisition time is predicted, and the global state vector set of the target object at the next data acquisition time is obtained.

[0019] Based on the global state vector set, shooting angle constraints, relative distance constraints between the UAV and the target object, and the position of the UAV at the current data acquisition time, the optimal observation point of the UAV at the next data acquisition time is obtained;

[0020] The drone is controlled to fly from its current location at the time of data acquisition to the target observation point at the next time of data acquisition.

[0021] The three-dimensional model of the target object is reconstructed using motion images collected by this UAV at the corresponding target observation point and motion images collected by other UAVs at the corresponding target observation point.

[0022] Thirdly, a computer device is provided, comprising a memory, a processor, and a transceiver connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive data, and the processor is used to read the computer program and execute a method for analyzing the correlation between oral and maxillofacial health and electronic health literacy as described in the first aspect.

[0023] Fourthly, a computer-readable storage medium is provided, on which instructions are stored, which, when executed on a computer, perform a method for reconstructing a three-dimensional model of a moving object based on multi-UAV collaborative operation as described in the first aspect.

[0024] Fifthly, a computer program product containing instructions is provided, which, when executed on a computer, causes the computer to perform a method for reconstructing a three-dimensional model of a moving object based on multi-UAV collaboration as described in the first aspect; the computer includes: a general-purpose computer, a special-purpose computer, or a programmable device.

[0025] Compared with existing technologies, this invention has the following advantages and beneficial effects: Through multi-UAV adaptive collaborative observation, trajectory prediction-driven adaptive perspective optimization, spatiotemporal alignment, and multi-level image fusion, high-precision, real-time 3D reconstruction of moving objects is achieved. The system can acquire image information from multiple angles, eliminating the influence of sampling time differences, illumination, and scale differences on reconstruction, thus improving model accuracy and completeness. Combining distributed Kalman filtering with deep neural network trajectory prediction ensures continuous acquisition of the best observation perspective in high-speed or complex motion scenarios, achieving high-precision, real-time 3D reconstruction of moving objects. The resulting high-precision 3D model can be directly applied to scenarios such as traffic monitoring, autonomous driving decision-making, and emergency dispatch, providing reliable support for accurate perception and intelligent decision-making in dynamic environments. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0027] Figure 1 This is a schematic diagram of the organizational structure of a three-dimensional model reconstruction system for moving objects based on multi-UAV collaborative operation, provided in Embodiment 1 of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. The illustrative embodiments and descriptions of this invention are for illustrative purposes only and are not intended to limit the invention. The embodiments described below are some, but not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] In the following description, numerous specific details are set forth to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that these specific details are not necessary to practice the invention. In other embodiments, well-known structures, materials, or methods are not specifically described to avoid obscuring the invention. Unless otherwise specified, the materials, instruments, and reagents used in the following embodiments are commercially available. Unless otherwise specified, the techniques used in the embodiments are conventional methods well known to those skilled in the art.

[0030] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0031] Example: A system for reconstructing 3D models of moving objects based on multi-UAV collaborative operation is provided, consisting of multiple UAVs. Each UAV is equipped with, for example, […]. Figure 1 The self-organizing network communication module, data acquisition module, time stamping module, state prediction module, trajectory prediction module, and flight control module are shown.

[0032] I. Self-organizing network communication module

[0033] The self-organizing network communication module is used to establish a two-way communication link between this drone and each of the other drones, forming a mesh network.

[0034] Multiple drones automatically connect and form a mesh network during missions, enabling communication sharing and laying the foundation for subsequent fusion of multi-source observation data. This mesh network ensures low-latency communication between drones, allowing each drone to broadcast its own real-time data while simultaneously receiving real-time data broadcast by other drones. In this way, information sharing is achieved across the entire drone swarm.

[0035] The self-organizing network communication module described in this embodiment can be a LoRa Mesh module (such as YL-800N and EWM528-2G4NW20SX) or a Wi-Fi Mesh module (such as the ESP32 series). It supports direct data transmission and has automatic routing, relay, broadcasting, and unicasting functions, enabling data pass-through and automatic relay between multiple nodes. When a drone equipped with a LoRa Mesh module or Wi-Fi Mesh module performs a mission, it automatically establishes a mesh network with other drones and broadcasts real-time data through the mesh network, while simultaneously receiving real-time data broadcast by other drones.

[0036] II. Data Acquisition Module

[0037] The data acquisition module is used to simultaneously acquire the flight status parameters of this UAV and the motion images of the target object.

[0038] The data acquisition module includes:

[0039] The real-time positioning unit is used to collect the real-time location of this UAV.

[0040] The real-time positioning unit can be selected from the DJI RTK module. The DJI RTK module is an important component of DJI that provides high-precision positioning for drones and other devices. It obtains coordinates and time through the DJI SDK, is plug-and-play, and is widely used in drones.

[0041] The speed acquisition unit is used to acquire the real-time flight speed of this UAV.

[0042] The velocity acquisition unit can be equipped with a GNSS Doppler frequency shift velocity sensor. By simultaneously tracking the frequency shift data of multiple satellites and combining it with satellite orbital parameters (ephemeris), the velocity vector of the UAV in three-dimensional space (eastward, northward, and celestial, i.e., ENU coordinate system) can be calculated.

[0043] The attitude monitoring unit is used to collect the real-time flight attitude of this UAV.

[0044] The attitude monitoring unit can be an inertial measurement unit (IMU). For example, the Bosch BMI088 low-noise gyroscope can provide stable angular velocity measurements under various conditions.

[0045] The image acquisition unit is used to acquire motion images of the target object.

[0046] The image acquisition unit can be equipped with the Stereolabs ZED 2i dual-view stereo camera, which, through stereo vision and neural network technology, can generate accurate depth maps in real time, enabling three-dimensional perception of the environment and can be used for object recognition, spatial positioning, etc.

[0047] The distance detection unit is used to collect the real-time distance between the UAV and the target object.

[0048] The distance detection unit can be selected from the LightWare SF30 / D laser rangefinder sensor.

[0049] III. Time stamp module

[0050] The time stamping module is used to mark the data acquisition time for flight status parameters and motion images.

[0051] The time stamp module can be equipped with a PTP / 1588 hardware clock, a precision clock synchronization device based on the IEEE 1588 protocol, which is mainly used to achieve high-precision time synchronization between various nodes in the network.

[0052] The process by which the time stamping module marks the data acquisition time for flight status parameters and motion images is as follows:

[0053] 1. Using GNSS 1PPS as the hardware second pulse, the local microcontroller of each UAV resets the local high-precision timer to zero through the PPS interrupt; at the same time, it reads the GNSS UTC second mark and records the current offset.

[0054] 2. Use PPS or flight controller TRIG to drive the camera's external trigger input. The camera returns a Frame Start / ExposureDone interrupt. The driver reads the local tick in the interrupt service routine and converts it to UTC. The UTC is then written to the frame metadata.

[0055] 3. The IMU performs phase alignment (phase lock) at a fixed frequency (≥200 Hz) on the rising edge of the PPS; each packet of IMU data records the start and end UTC of sampling.

[0056] 4. Select one GNSS-equipped UAV / ground station in the mesh network as the master clock; perform BC / OC on other nodes; use RF MAC hard timestamps for PTP alignment to stabilize the cross-machine clock deviation within the range of hundreds of microseconds to milliseconds.

[0057] 5. Estimate the residual drift between the camera / IMU / GNSS; use UTC and perform time alignment and interpolation before fusion, and pass the time uncertainty into the filter weights.

[0058] The method of using a timestamp module to mark the data acquisition time of flight status parameters and moving images can also be found in "Hardware Timestamping for an Image Acquisition System Based on FlexRIO and IEEE 1588 v2 Standard," published in IEEE Transactions on Nuclear Science. This paper introduces a hardware timestamping implementation scheme for an image acquisition system based on the FlexRIO and IEEE 1588 v2 standards. This system utilizes a Field-Programmable Gate Array (FPGA) and a timer card synchronized via the PTP v2 protocol to achieve synchronization with the master clock, image acquisition and processing, and a timestamp generation mechanism.

[0059] IV. State Prediction Module

[0060] The state prediction module is used to predict the motion state of the target object at the next data acquisition time based on the flight state parameters of the UAV at the current data acquisition time and the motion image of the target object at the current data acquisition time, so as to obtain the global state vector set of the target object at the next data acquisition time.

[0061] In multi-UAV collaborative observation, the images and sensor data acquired by each UAV are localized and uncertain, manifesting as follows: different UAVs shooting from different angles lead to measurement deviations; ambient lighting, sensor noise, and partial occlusion by moving objects all affect the accuracy of observations; and data from a single UAV is insufficient to comprehensively describe the velocity and position of moving objects. Therefore, a distributed filtering method is employed to allow each UAV to retain its own observation data while sharing motion state prediction results with other UAVs, and gradually fusing them within a mesh network to obtain a globally consistent estimate of the target object's motion state. The core idea is: each UAV processes its own locally acquired observation data using a filtering method (such as Extended Kalman Filter (EKF) or Unscented Kalman Filter (UKF); based on the processed local observation data, it predicts the target object's motion state, generating local prediction results; and a distributed information fusion algorithm with consistency constraints propagates and weights the local prediction results within the mesh network to obtain a global prediction result for the target object.

[0062] The state prediction module includes:

[0063] (a) Observation Vector Generation Unit

[0064] The observation vector generation unit is used to generate the observation vector of the target object at the current data acquisition time based on the flight status of the UAV at the current data acquisition time and the motion image of the target object at the current data acquisition time.

[0065] The observation vector generation unit includes:

[0066] 1. Target positioning subunit

[0067] The target localization subunit is used to locate target objects in the motion image at the current data acquisition time using a deep learning target detection model, and generate a two-dimensional bounding box of the target object.

[0068] The target localization subunit can be selected from NVIDIA Retson, Ascend 310 chips, or edge GPU modules. The target localization subunit uses a deep learning object detection model to locate target objects in moving images; therefore, a deep learning object detection model needs to be pre-deployed on the target localization subunit. The deep learning object detection model can be either a YOLOv5 model or a YOLOv8 model.

[0069] Taking the YOLOv5 model as an example: The process of locating a target object in a motion image at the current data acquisition time using the YOLOv5 model is as follows:

[0070] (1) Image preprocessing. Scale the acquired motion image to the resolution required by the YOLOv5 model (e.g., 640×640); normalize the pixels of the scaled motion image (e.g., normalize the pixel values ​​to the 0~1 range); convert the normalized image into tensor format.

[0071] (2) Image feature extraction. Multidimensional features are extracted from the preprocessed motion image using the backbone network of the YOLOv5 model. Multidimensional features include edge features, texture features, shape features, and semantic features.

[0072] (3) Multidimensional feature fusion. Multidimensional features are fused through the neck network of the YOLOv5 model to generate feature maps.

[0073] (4) Object Detection. Candidate boxes are generated directly on the feature map using the detection head of the YOLOv5 model, and the target category and confidence score are predicted. Finally, non-maximum suppression (NMS) is used to filter out candidate boxes with confidence scores less than a threshold, and the detection results are output. The output detection results include: the two-dimensional bounding box of the target object, the target category, and the confidence score.

[0074] The above-described method of using the YOLOv5 model to locate target objects is existing technology. For further information on using the YOLOv5 model to locate target objects, please refer to the published document of invention patent application CN202211278019.3, entitled "An Improved YOLOv5 Target Detection Method Applicable to Low-Light Environments," which will not be elaborated upon in this embodiment.

[0075] By employing a deep learning-based object detection model, UAVs can quickly and accurately identify and locate target objects in complex environments, overcoming the limitations of traditional edge detection / color thresholding methods. Even under varying lighting conditions, complex backgrounds, or partial target occlusion, the model maintains high detection accuracy. The output can be seamlessly embedded into subsequent sensor fusion and distributed filtering processes, improving the accuracy of 3D modeling of moving objects.

[0076] 2. Feature Extraction Subunit

[0077] The feature extraction subunit is used to extract multiple feature points of the target object from the subgraph corresponding to the two-dimensional bounding box using a feature extraction algorithm, and obtain a set of feature points.

[0078] The target object will undergo scale changes, rotation, lighting changes, and even partial occlusion at different time frames and from different UAV perspectives. In order to achieve stable tracking of the target object under these conditions, it is necessary to extract feature points of the target object that remain stable and discriminative under different scales, rotations, and lighting conditions, so as to facilitate stable tracking of the target object.

[0079] Similarly, the feature extraction subunit can be an NVIDIA Retson, Ascend 310 chip, or edge GPU module, and a feature point extraction algorithm needs to be loaded onto the feature extraction subunit. The feature point extraction algorithm can be SIFT, SURF, or ORB.

[0080] Taking the SIFT algorithm as an example, the process of extracting multiple feature points of the target object from the sub-image corresponding to the 2D bounding box is as follows:

[0081] (1) Image preprocessing. The sub-images are converted to grayscale to reduce computational overhead, and noise in the grayscale images is removed by Gaussian filtering or bilateral filtering.

[0082] (2) Feature point extraction. The SIFT algorithm is used to extract multiple feature points of the target object from the preprocessed grayscale image, and the feature descriptor for each feature point is calculated. The SIFT algorithm's feature point extraction process includes:

[0083] Key point detection—Searching for local extrema in the preprocessed grayscale image and filtering out local extrema with contrast less than a threshold.

[0084] Feature point selection – Using non-maximum suppression, local extrema (redundant points) with a distribution density greater than a threshold are removed, and the RANSAC algorithm (random sample consensus algorithm) is used to remove points with a matching error greater than an error threshold. The remaining local extrema are then used as feature points.

[0085] Feature point description—Calculate the feature descriptor for each feature point (e.g., generate a 64-dimensional or 128-dimensional floating-point feature vector).

[0086] The above-described method for extracting feature points using the SIFT algorithm is existing technology. Other real-time methods for extracting feature points using the SIFT algorithm can be found in the publication document of invention patent application CN201510091082.X, entitled "A SIFT Algorithm for Image Matching," which will not be elaborated upon in this embodiment.

[0087] The extracted feature points have good invariance and stability, ensuring that they can still be correctly identified and matched under multiple drones and multiple viewpoints, reducing observation errors caused by lighting, rotation, and occlusion, and making subsequent distributed filtering estimation, image stitching, and 3D modeling more accurate and robust.

[0088] (3) Generate a set of feature points—Associate feature descriptors with their corresponding feature points to generate a set of feature point coordinates. The set of feature point coordinates is as follows: F i ( t )={( u 1, v 1),( u 2, v 2),…,( u r , v r ),…,( u m , v m )}.in, F i Indicates the first i The feature point set corresponding to each drone r The feature point number is the feature point number in the feature point set. m The total number of feature points in the feature point set. r =1,2,…, m , u r For the first r The x-coordinates of the feature points v r For the first r The ordinates of each feature point carry feature descriptor information.

[0089] 3. Global Mapping Subunit

[0090] The global mapping subunit is used to map the two-dimensional coordinate vector of each feature point in the feature point set to the camera coordinate system using the camera intrinsic parameters, to obtain the ray direction vector corresponding to each two-dimensional coordinate vector. Then, it uses the camera extrinsic parameters to map each ray direction vector to the UAV body coordinate system, to obtain the three-dimensional coordinate vector corresponding to each ray direction vector. Finally, it uses the flight attitude parameters of the UAV to map each three-dimensional coordinate vector to the world coordinate system. Then, it uses the camera depth value to map each three-dimensional coordinate vector in the world coordinate system to the global coordinate system, to obtain the global three-dimensional coordinate vector corresponding to each three-dimensional coordinate vector. Finally, it stores each global three-dimensional coordinate vector into a global three-dimensional coordinate vector set.

[0091] Since the coordinates of feature points extracted solely from camera images are local two-dimensional pixel coordinates, without pose compensation, feature points extracted from moving images captured by different drones will reside in their respective independent camera coordinate systems, making alignment in the global coordinate system impossible and hindering 3D reconstruction of moving objects. Therefore, it is necessary to map the coordinates of all feature points to a unified global coordinate system. The process of mapping feature point coordinates to the global coordinate system is as follows:

[0092] (1) Map the coordinates of each feature point to the camera coordinate system to establish a set of ray direction vectors. The mapping relationship is as follows: .in, P cam Let be the ray direction vector in the camera coordinate system. x c The x-coordinate of the feature point in the camera coordinate system. y c The ordinate of the feature point in the camera coordinate system. K For the camera intrinsic parameter matrix, u The x-coordinate of the feature point in the image. v This represents the ordinate of the feature point in the image. (Camera intrinsic parameter matrix) ,in, f x The focal length is in the width direction. f y The focal length in the height direction. c x The x-coordinate of the principal point c y The ordinate of the main point.

[0093] (2) Map each ray direction vector to the UAV body coordinates to establish a three-dimensional coordinate vector set. The mapping relationship is as follows: .in, p uavThis is a three-dimensional coordinate vector in the UAV's body coordinate system; R cam Let be the rotation matrix of the camera coordinate system relative to the UAV body coordinate system. R cam The external parameters for the fixed installation of the camera and the aircraft are obtained through calibration. That is, the attitude monitoring module of the UAV provides the attitude information of the aircraft in real time. By combining the attitude of the aircraft with the external parameter matrix, the rotation relationship matrix of the camera coordinate system relative to the geographic coordinate system can be calculated in real time. t cam This is the translation vector of the camera coordinate system origin relative to the UAV body coordinate system origin, which can be obtained by performing camera-UAV calibration in advance.

[0094] (3) Map the three-dimensional coordinate vectors of each UAV body coordinate system to the world coordinate system, and establish a set of three-dimensional coordinate vectors in the world coordinate system. The mapping relationship is as follows: .in, p world A three-dimensional coordinate vector in the world coordinate system. R imu The attitude rotation matrix of the UAV (provided by the attitude monitoring module IMU). p uav This is a three-dimensional coordinate vector in the UAV coordinate system. X uav The x-coordinate of the UAV in the world coordinate system (provided by GPS). Y uav The vertical coordinate of the UAV in the world coordinate system (provided by GPS). Z uav The normal coordinates of the UAV in the world coordinate system (provided by GPS).

[0095] (4) Map the set of three-dimensional coordinate vectors in the world coordinate system to the set of global three-dimensional coordinate vectors. The mapping relationship is as follows: .in, P world It is a global set of three-dimensional coordinate vectors. Z This represents the camera's depth value. Since the image acquisition module selected in this embodiment is a stereo camera, the camera's depth value can be calculated using the stereo camera's baseline length and parallax. .in, f For the camera's focal length, B The baseline length of the camera. d The parallax of the camera, baseline length, and parallax can be obtained through camera calibration.

[0096] Ultimately, the coordinates of each feature point in the feature point coordinate set are mapped to a unified global coordinate system. This allows feature points acquired by all UAVs to be superimposed in the same 3D space for 3D reconstruction of moving objects. By mapping the feature point coordinate set to the global coordinate system, attitude differences between different UAVs and at different time frames are eliminated, ensuring the consistency of feature points in the global space. Position compensation provided by GPS enables cross-UAV observation data alignment, improving the accuracy and robustness of 3D reconstruction of moving objects.

[0097] 4. Speed ​​Acquisition Subunit

[0098] The velocity acquisition subunit is used to extract the global three-dimensional coordinate vector of any feature point at adjacent data acquisition times from the global three-dimensional coordinate vector set, and obtain the velocity vector of the target object based on the extracted global three-dimensional coordinate vector.

[0099] Assuming at the time of data acquisition t k , No. r The global 3D coordinates of the feature points are [ p r,x ( k ), p r,y ( k ), p r,z ( k )].in, p r,x ( k (At the time of data acquisition) t k , No. r Feature points x Axis coordinate components; p r,y ( k (At the time of data acquisition) t k , No. r Feature points y Axis coordinate components; p r,z ( k (At the time of data acquisition) t k , No. r Feature points z Axis coordinate components. Assuming at the data acquisition time... t k-1 , No. r The global 3D coordinates of the feature points are [ p r,x ( k- 1), pr,y ( k- 1), p r,z ( k- 1)]. Among them, p r,x ( k- 1) For data acquisition time t k-1 , No. r Feature points x Axis coordinate components; p r,y ( k- 1) For data acquisition time t k-1 , No. r Feature points y Axis coordinate components; p r,z ( k- 1) For data acquisition time t k , No. r Feature points z Axis coordinate components.

[0100] Then the first r Each feature point in x The velocity component of the shaft is v r,x ( k )=[ p r,x ( k )- p r,x ( k- 1)] / ( t k - t k-1 Similarly, the number of... r Each feature point in y The velocity components of the shaft and in z The velocity component of the axis. At the time of data acquisition. t k , No. r The velocity components of a feature point can be expressed as [ v r,x ( k ), v r,y ( k ), v r,z ( k )).

[0101] It should be noted that: due to the time of data acquisition t kEach feature point of the target object in x The velocity components of the shaft are all the same (in y The velocity components of the shaft and in z The velocity component of the axis is similar (therefore, the data acquisition time can be...). t k All feature points in x The velocity components of the shaft are uniformly represented as v x ( k Similarly, the data acquisition time can be... t k All feature points in y The velocity components of the shaft and in z The velocity components of the shaft are uniformly represented as follows: v y ( k )and v z ( k Then at the time of data acquisition. t k The velocity vector of any feature point of the target object can be expressed as v =[ v x ( k ), v y ( k ), v z ( k )).

[0102] 5. Vector Generating Subunit

[0103] The vector generation unit is used to combine the global three-dimensional coordinate vector with the velocity vector to obtain the global observation vector for each feature point and establish a global observation vector set.

[0104] For example, at the time of data acquisition t k The first of the target objects r The global observation vector of a feature point can be represented as: c r ( k )=[ p r,x ( k ), p r,y ( k ), p r,z ( k ), v x ( k ), vy ( k ), v z ( k )).

[0105] (ii) Motion state prediction unit

[0106] The motion state prediction unit is used to predict the motion state of each feature point of the target object at the next data acquisition moment based on the global observation vector set, establish a state vector set, and call the ad hoc network communication module to broadcast the state vector set of this UAV through the mesh network, and receive the state vector set broadcast by each of the other UAVs through the mesh network.

[0107] This embodiment uses a uniform velocity model to predict the motion state of feature points at the next data acquisition time, and calculates the covariance matrix to calibrate the uncertainty of the predicted motion state.

[0108] (1) The expression for the uniform velocity model is: .in, s r ( k +1) indicates the first r Each feature point at the data acquisition time t k+1 The state vector, F Here is the state transition matrix. c r ( k ) is the first of the target objects r Global observation vector of each feature point This is process noise.

[0109] It should be noted that since the acceleration of the target object between two adjacent data acquisition moments can be considered zero, a uniform velocity model can be used to characterize the motion state of the target object.

[0110] Furthermore, s r ( k +1) contains the target object's first r The position coordinates and velocity of each feature point at the next data acquisition time are then... s r ( k +1)=[ p’ r,x ( k +1), p’ r,y ( k +1), p’ r,z ( k +1), vx ( k +1), v y ( k +1), v z ( k +1)]. Among them, p’ r,x ( k +1) indicates the time of data acquisition. t k+1 , No. r The global 3D coordinates of the feature points are in x The coordinate components of the axes; p’ r,y ( k +1) indicates the time of data acquisition. t k+1 , No. r The global 3D coordinates of the feature points are in y The coordinate components of the axes; p’ r,z ( k +1) indicates the time of data acquisition. t k+1 , No. r The global 3D coordinates of the feature points are in z The coordinate components of the axes; v x ( k +1) indicates the time of data acquisition. t k+1 , No. r Each feature point in x The velocity component of the shaft; v y ( k +1) indicates the time of data acquisition. t k+1 , No. r Each feature point in y The velocity component of the shaft; v z ( k +1) indicates the time of data acquisition. t k+1 , No. r Each feature point in z The velocity component of the shaft.

[0111] , This represents the time difference between the current data acquisition moment and the next data acquisition moment.

[0112] Furthermore, in reality, the motion of a target object rarely perfectly conforms to an ideal uniform velocity model and may be affected by various uncertainties (such as wind force, changes in ground friction, etc.). These interfering factors are referred to as process noise. To represent. Process noise. It is usually assumed to be Gaussian noise with zero mean, which is used to simulate the deviation between the model and the actual motion, and to ensure the robustness of the model.

[0113] Based on the state vector of each feature point predicted by the above uniform velocity model at the next data acquisition time, the set of state vectors of the entire target object at the next data acquisition time is obtained.

[0114] (2) The formula for calculating the covariance matrix is: .in, P r ( k+ 1) is the first r Each feature point at the data acquisition time t k+1 The covariance matrix, P r ( k ) is the first r Each feature point at the data acquisition time t k The covariance matrix, F T This is the transpose of the state transition matrix. Q Let be the process noise covariance matrix.

[0115] It should be noted that: 1) the first r The covariance matrix of the feature points at the initial time is:

[0116] ,in, This indicates that at the initial moment, the first... r The global 3D coordinates of the feature points are in x The standard deviation of the coordinate components of the axes. This indicates that at the initial moment, the first... r The global 3D coordinates of the feature points are in y The standard deviation of the coordinate components of the axes. This indicates that at the initial moment, the first... r The global 3D coordinates of the feature points are in z The standard deviation of the coordinate components of the axes. This indicates that at the initial moment, the first... r Each feature point in x The standard deviation of the velocity components of the shaft, This indicates that at the initial moment, the first... r Each feature point in yThe standard deviation of the velocity components of the shaft, This indicates that at the initial moment, the first... r Each feature point in z The standard deviation of the velocity components of the shaft.

[0117] 2) Acceleration noise covariance Q This can be obtained based on historical data statistics. Specifically, the standard deviation of the acceleration change is calculated from the historical velocity of the target object, the variance is calculated from the standard deviation, and the process noise covariance matrix is ​​established using the variance. Q The variance lies on the diagonal of the covariance matrix.

[0118] (III) State Vector Fusion Unit

[0119] The state vector fusion unit is used to update the state vector set of the UAV by weighted averaging based on the state vector set of the UAV and the received state vector set, so as to obtain the global state vector set.

[0120] Because different UAVs have different observation conditions, locations, and sensor accuracies, the reliability of the same vector predicted by each UAV varies. Global estimation needs to consider the weight of each local estimate, so that the estimate with less uncertainty accounts for a larger proportion, ultimately resulting in a convergent and consistent global state. The weighted average processing is as follows:

[0121] 1. For each feature point, calculate the first covariance using the corresponding covariance matrix. i Taiwanese drones and the first j The weights among the drones are determined, and a weight matrix is ​​established for each feature point.

[0122] The formula for calculating the weight is: .in, Indicates that for the first r The feature point, the first i Taiwanese drones and the first j The weighting of drones among different drones; i and j Number the drone. i ∈ N , j ∈ N , N Indicates a drone swarm; Indicates the first i Among the target objects corresponding to the drones, the first r Each feature point at the data acquisition time t k+1 The inverse of the covariance matrix; Indicates the first i Among the target objects corresponding to the drones, the first rEach feature point at the data acquisition time t k+1 The determinant of the inverse of the covariance matrix; Indicates the first j Among the target objects corresponding to the drones, the first r Each feature point at the data acquisition time t k+1 The inverse of the covariance matrix; Indicates the first j Among the target objects corresponding to the drones, the first r Each feature point at the data acquisition time t k+1 The determinant of the inverse of the covariance matrix.

[0123] 2. Update the state vector of each feature point using the corresponding weight matrix. The state vector update model is as follows: .in, For the first i The first drone extracted r Each feature point at the data acquisition time t k+1 The global motion state vector, For the first r The weight matrix corresponding to each feature point For the first i The first drone extracted r Each feature point at the data acquisition time t k+1 The state vector, For the first j The first drone extracted r Each feature point at the data acquisition time t k+1 The state vector.

[0124] By fusing state vectors to make data from multiple UAVs complementary, the system can still obtain stable estimation results even if the observations of individual UAVs are disturbed. In addition, distributed filtering avoids the network bottleneck caused by centralized processing. Each UAV can process data in parallel, reducing latency. Furthermore, as the number of UAVs increases, the distributed structure can naturally expand, further improving estimation accuracy. Compared with single-UAV observations, the position and velocity estimation errors after distributed filtering fusion are significantly reduced, making subsequent trajectory prediction more reliable.

[0125] V. Location Acquisition Module

[0126] The point acquisition module is used to obtain the best observation point of the UAV at the next data acquisition time based on the global state vector set, shooting angle constraints, relative distance constraints between the UAV and the target object, and the position of the UAV at the current data acquisition time.

[0127] When multiple drones photograph and reconstruct a 3D image of a target object, each drone needs to determine its optimal relative position to the object. Without a proper relative position, problems may arise: the drone may be too far from the target, resulting in low image resolution; or the drone may be too close, leading to the loss of part of the target image and image distortion. Therefore, it is necessary to dynamically plan the optimal positional relationship between the drone and the target object in terms of relative distance and observation angle.

[0128] The location acquisition module includes:

[0129] (a) Unit for obtaining center point coordinates

[0130] The center point coordinate acquisition unit is used to obtain the center point coordinate vector of the target object at the next data acquisition moment based on the global observation vector set.

[0131] The center point coordinate vector can be calculated using the arithmetic mean method. For example, the feature extraction subunit extracts the target object's... N Each feature point is used to obtain the next data acquisition time (e.g., 1 feature point) through the motion state prediction unit. t k+1 Given the state vector (which includes position coordinates) at time ( ), the coordinates of the center point of the target object at the next data acquisition time, calculated using the arithmetic mean method, are: The coordinate vector of the center point of the target object at the next data acquisition moment is: C obj =( C x , C y , C z ).

[0132] (ii) Vertical feasible region acquisition unit

[0133] The vertical feasible region acquisition unit is used to obtain the vertical feasible region based on the shooting angle constraint, relative distance constraint, and center point coordinate vector. z The axis component is used to obtain the vertical feasible region of this UAV at the next data acquisition moment.

[0134] It should be noted in advance that:

[0135] The shooting angle constraint mentioned in this embodiment refers to the angular difference (i.e., pitch angle) between the normal direction of the target object and the optical axis of the camera. i The allowable fluctuation range; the allowable fluctuation range is expressed as [ i min , i max ],in, i minThis is the minimum pitch angle. i max This represents the maximum pitch angle.

[0136] The relative distance constraint described in this embodiment refers to the adjustable range of the straight-line shooting distance of the UAV relative to the target object while ensuring the resolution of the target object in the moving image.

[0137] The relative distance adjustment range is expressed as: [ d min , d max ].in, d min This represents the minimum shooting distance of the drone relative to the target object. d max This represents the maximum shooting distance of the drone relative to the target object. Based on the principle of similar triangles in camera imaging, we can derive: , .in, D This represents the actual width of the target object in the world coordinate system. S max The maximum value of the projected pixels. S min This represents the minimum desired projected pixel value.

[0138] Furthermore, the actual width of the target object in the world coordinate system. D It can be calculated from the global state vector set. Specifically, it is found from the global state vector set in... x The maximum and minimum values ​​of the coordinate components of the axes are calculated in... x The difference between the maximum and minimum values ​​of the coordinate components of the axes gives the actual width of the target object in the world coordinate system. D Additionally, the minimum desired projected pixel value. S min This refers to: the minimum pixel width that a target object should occupy in a moving image captured by a drone; and the expected maximum projected pixel value. S max This refers to the maximum pixel width that a target object must occupy in a moving image captured by a drone. For example, in a real-world project, if the target object is required to occupy at least 80 pixels wide and no more than 1200 pixels wide in the moving image, then... S min =80px, S max =1200px.

[0139] Based on the above constraints on shooting angle and relative distance, the method for obtaining the vertical feasible region is explained below:

[0140] At candidate observation pointsU uav =( U x , U y , U z At point ), pitch angle i Satisfies the following trigonometric function relationships: , .in, U z The three-dimensional coordinate vector of the candidate observation point z Axial components; d This represents the straight-line distance between the candidate observation point and the center point of the target object at the next data acquisition time. d ∈[ d min , d max ]; L This represents the horizontal distance between the candidate observation point and the center point of the target object at the next data acquisition time. ; U x The three-dimensional coordinate vector of the candidate observation point x Axial components, U y The three-dimensional coordinate vector of the candidate observation point y Axial components.

[0141] Due to pitch angle i There is an allowable fluctuation range. i min , i max ], and the height of the target object C z Given this, it can be deduced from the range of values ​​for the vertical height difference. U z The range of values ​​for (i.e., the vertical feasible region). Specifically, ,in, This represents the vertical height difference between the candidate observation point and the center point of the target object at the next data acquisition time. Once the pitch angle is determined... i and straight-line distance d The three-dimensional coordinates of the candidate observation points z The axial component is thus uniquely determined.

[0142] (III) Horizontal Feasible Region Acquisition Unit

[0143] The horizontal feasible region acquisition unit is used to determine the feasible region based on shooting angle constraints, relative distance constraints, and center point coordinate vectors. x Axis components and center point coordinate vectorsy The axis component is used to obtain the horizontal feasible region of this UAV at the next data acquisition moment.

[0144] Similarly, based on the shooting angle constraints and relative distance constraints mentioned above, the method for obtaining the horizontal feasible region is explained below:

[0145] exist xyz In the plane (height locked), the horizontal distance between the candidate observation point and the center point of the target object at the next data acquisition time. L Pitch angle is possible i distance from the line d Derivation, i.e. Therefore, in xyz In the plane, the horizontal coordinates of the candidate observation points ( U x , U y ) must meet This is the horizontal coordinate of the center point of the target object. C x , C y (with the center as the center) L A circle with radius .

[0146] At the same time, straight-line distance d Must meet d ∈[ d min , d max Therefore, the horizontal distance L There is also a corresponding range of values: , cos i Follow i It decreases as it increases. Ultimately, xyz The horizontal feasible region in the plane is an annulus between two concentric circles: the radius of the inner circle is... L min outer circle radius L max The center of the circle is ( C x , C y ).

[0147] (iv) Three-dimensional feasible region generation unit

[0148] The 3D feasible domain generation unit is used to generate the 3D feasible domain of the UAV at the next data acquisition moment based on the vertical and horizontal feasible domains.

[0149] (v) Optimal Observation Point Selection Unit

[0150] The optimal observation point selection unit is used to select the point closest to the location of the UAV at the current data acquisition time within the three-dimensional feasible domain as the optimal observation point.

[0151] The core of selecting the optimal observation point is to find the point that is closest to the coordinates of the UAV at the current data acquisition time within the above three-dimensional feasible area.

[0152] Coordinates of the drone at the current data acquisition moment U now =(U x,now U y,now U z,now ) and candidate observation points U uav =( U x , U y , U z The distance is .

[0153] VI. Flight Control Module

[0154] The flight control module is used to control the UAV to fly from its current location at the time of data acquisition to the target observation point.

[0155] VII. 3D Reconstruction Module

[0156] The 3D reconstruction module is used to reconstruct the 3D model of the target object using motion images collected by this UAV at the corresponding target observation point and motion images collected by other UAVs at the corresponding target observation point.

[0157] For methods to reconstruct 3D models of target objects based on motion images acquired from multiple angles, see US10198858B2, "Method for 3D modelling based on structure from motionprocessing of sparse 2D images," which details the apparatus and process for generating 3D models by processing sparse photographic image sequences from different angles. Also, US20180160102A1, "METHODFOR 3D RECONSTRUCTION OF AN ENVIRONMENT OF A MOBILE DEVICE, CORRESPONDING COMPUTERPROGRAM PRODUCT AND DEVICE," discloses a multi-view 3D reconstruction method for mobile devices / cameras. This method proposes performing a coarse reconstruction followed by a multi-view photometric / refinement step (e.g., multi-view photometric stereo) to refine the 3D representation of local targets, including time-series image processing and block reconstruction procedures.

[0158] Example 2: Corresponding to Example 1, this example provides a method for reconstructing a 3D model of a moving object based on multi-UAV collaboration. Based on multiple UAVs, the following steps are performed on each UAV:

[0159] Step 1: Establish a two-way communication link between this drone and every other drone to form a mesh network;

[0160] Step 2: Synchronously collect the flight status parameters of this UAV and the motion image of the target object, add timestamps to the flight status parameters and motion images, and call the self-organizing network communication module to transmit the flight status parameters, motion images and corresponding timestamps through the mesh network;

[0161] Step 3: Based on the flight state parameters of the UAV at the current data acquisition time and the motion image of the target object at the current data acquisition time, predict the motion state of the target object at the next data acquisition time, and obtain the global state vector set of the target object at the next data acquisition time.

[0162] Step 4: Based on the global state vector set, shooting angle constraints, relative distance constraints between the UAV and the target object, and the position of the UAV at the current data acquisition time, obtain the optimal observation point of the UAV at the next data acquisition time;

[0163] Step 5: Control the UAV to fly from its current location at the data acquisition time to the target observation point at the next data acquisition time;

[0164] Step 6: Reconstruct the 3D model of the target object using motion images collected by this UAV at the corresponding target observation point and motion images collected by other UAVs at the corresponding target observation points.

[0165] The flight status parameters include: the real-time position, real-time flight speed, and real-time flight attitude of the UAV.

[0166] Furthermore, obtaining the global state vector set of the target object at the next data acquisition moment includes the following steps:

[0167] Based on the flight status parameters of this UAV at the current data acquisition time and the motion image of the target object at the current data acquisition time, generate a global observation vector for each feature point of the target object at the current data acquisition time, and establish a global observation vector set;

[0168] Based on the global observation vector set, the motion state of each feature point of the target object at the next data acquisition moment is predicted, a state vector set is established, and the self-organizing network communication module is called to broadcast the state vector set of this UAV through the mesh network, and to receive the state vector set broadcast by each of the other UAVs through the mesh network.

[0169] Based on the state vector set of this UAV and the received state vector set, the state vector set of this UAV is updated by weighted averaging to obtain the global state vector set.

[0170] Furthermore, a global observation vector set is established, including the following steps:

[0171] The target object in the motion image at the current data acquisition time is located by a deep learning object detection model, and a two-dimensional bounding box of the target object is generated.

[0172] Multiple feature points of the target object are extracted from the sub-image corresponding to the two-dimensional bounding box using a feature extraction algorithm, resulting in a set of feature points.

[0173] Using the camera intrinsic parameters, the two-dimensional coordinate vector of each feature point in the feature point set is mapped to the camera coordinate system to obtain the ray direction vector corresponding to each two-dimensional coordinate vector. Using the camera extrinsic parameters, each ray direction vector is mapped to the UAV body coordinate system to obtain the three-dimensional coordinate vector corresponding to each ray direction vector. Using the flight attitude parameters of this UAV, each three-dimensional coordinate vector is mapped to the world coordinate system. Using the camera depth value, each three-dimensional coordinate vector in the world coordinate system is mapped to the global coordinate system to obtain the global three-dimensional coordinate vector corresponding to each three-dimensional coordinate vector. Each global three-dimensional coordinate vector is stored in the global three-dimensional coordinate vector set.

[0174] Extract the global three-dimensional coordinate vector of any feature point at adjacent data acquisition times from the global three-dimensional coordinate vector set, and obtain the velocity vector of each feature point based on the extracted global three-dimensional coordinate vector;

[0175] By combining the global 3D coordinate vector with the velocity vector, the global observation vector of each feature point is obtained, and a global observation vector set is established.

[0176] Furthermore, to obtain the optimal observation point for this UAV at the next data acquisition time, the following steps are included:

[0177] Obtain the center point coordinate vector of the target object at the next data acquisition moment based on the global observation vector set;

[0178] Based on the shooting angle constraint, relative distance constraint, and center point coordinate vector z The axis component is used to obtain the vertical feasible region of this UAV at the next data acquisition moment;

[0179] Based on shooting angle constraints, relative distance constraints, and center point coordinate vector x Axis components and center point coordinate vectors y The axis component is used to obtain the horizontal feasible region of this UAV at the next data acquisition moment;

[0180] The three-dimensional feasible domain of the cost UAV at the next data acquisition moment is generated based on the vertical and horizontal feasible domains.

[0181] Within the three-dimensional feasible domain, select the point closest to the location of the UAV at the current data acquisition time as the target observation point.

[0182] Example 3: Based on the method provided in Example 1 and the system provided in Example 2, this example provides a computer device that executes the method described in Example 2 or any other method that may involve the method described in Example 2. The device includes a memory, a processor, and a transceiver connected in sequence. The memory stores a computer program, the transceiver sends and receives messages, and the processor reads the computer program and executes the method described in Example 2 or any other method that may involve the method described in Example 2. Specifically, the memory may include, but is not limited to, random-access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out (FIFO) memory, and / or last-in-first-out (FILO) memory, etc.; the processor may include, but is not limited to, an STM32F105 series microprocessor. Furthermore, the computer device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0183] The working process, working details and technical effects of the aforementioned computer device provided in this embodiment can be found in the method described in Embodiment 1 or any method that may involve the method described in Embodiment 2, and will not be repeated here.

[0184] Example 4: This example provides a computer-readable storage medium that stores instructions that include the method described in Example 2 or any other method that may involve the method described in Example 2. Specifically, the computer-readable storage medium stores instructions that, when executed on a computer, perform the method described in Example 2 or any other method that may involve the method described in Example 2. The computer-readable storage medium refers to a data storage medium, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or Memory Sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0185] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in this embodiment can be found in the method described in Embodiment 2 or any method that may be related to Embodiment 2, and will not be repeated here.

[0186] Example 5: This example provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform the method described in Example 2 or any method that may involve the method described in Example 2. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0187] It should be understood that the terms "system," "device," "unit," and / or "module" as used in this specification are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0188] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0189] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0190] It should be noted that the structures, proportions, sizes, etc., illustrated in the accompanying drawings are merely for illustrative purposes to aid those skilled in the art and are not intended to limit the scope of the invention. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in proportions, or adjustments to size, without affecting the effectiveness and purpose of the invention, should still fall within the scope of the disclosed technical content. Furthermore, terms such as "upper," "lower," "left," "right," and "middle" used in this specification are merely for clarity and not intended to limit the scope of the invention. Changes or adjustments to their relative relationships, without substantially altering the technical content, should also be considered within the scope of the invention.

Claims

1. A system for reconstructing a three-dimensional model of a moving object based on multi-unmanned aerial vehicle (UAV) cooperation, characterized in that, Including multiple drones; each drone carries: The self-organizing network communication module is used to establish a two-way communication link between this drone and each of the other drones to form a mesh network; The data acquisition module is used to synchronously acquire the flight status parameters of this UAV and the motion images of the target object, add timestamps to the flight status parameters and motion images, and call the self-organizing network communication module to broadcast the flight status parameters, motion images and corresponding timestamps through the mesh network; The state prediction module is used to predict the motion state of the target object at the next data acquisition time based on the flight state parameters of the UAV at the current data acquisition time and the motion image of the target object at the current data acquisition time, so as to obtain the global state vector set of the target object at the next data acquisition time. The point acquisition module is used to obtain the best observation point of the UAV at the next data acquisition time based on the global state vector set, shooting angle constraints, relative distance constraints between the UAV and the target object, and the position of the UAV at the current data acquisition time. The flight control module is used to control the UAV to fly from its current location at the time of data acquisition to the target observation point at the next time of data acquisition. The 3D reconstruction module is used to reconstruct the 3D model of the target object using motion images collected by this UAV at the corresponding target observation point and motion images collected by other UAVs at the corresponding target observation point. The state prediction module includes: The observation vector generation unit is used to generate a global observation vector for each feature point of the target object at the current data acquisition time based on the flight state parameters of the UAV at the current data acquisition time and the motion image of the target object at the current data acquisition time, and to establish a global observation vector set. The motion state prediction unit is used to predict the motion state of each feature point of the target object at the next data acquisition moment based on the global observation vector set, establish a state vector set, and call the self-organizing network communication module to broadcast the state vector set of this UAV through the mesh network, and receive the state vector set broadcast by each other UAV through the mesh network. The state vector fusion unit is used to update the state vector set of the UAV by weighted averaging based on the state vector set of the UAV and the received state vector set, so as to obtain the global state vector set. 2.The multi-UAV cooperative based moving object 3D model reconstruction system of claim 1, wherein, The data acquisition module includes: The real-time positioning unit is used to collect the real-time position of this UAV; The speed acquisition unit is used to acquire the real-time flight speed of this UAV; Attitude monitoring unit, used to collect the real-time flight attitude of this UAV; The image acquisition unit is used to acquire motion images of the target object; The distance detection unit is used to collect the real-time distance between the UAV and the target object. 3.The multi-UAV coordinated based moving object 3D model reconstruction system of claim 1, wherein, The observation vector generation unit includes: The target localization subunit is used to locate target objects in the motion image at the current data acquisition time through a deep learning target detection model and generate a two-dimensional bounding box of the target object. The feature extraction subunit is used to extract multiple feature points of the target object from the sub-graph corresponding to the two-dimensional bounding box using a feature extraction algorithm, and obtain a set of feature points. The global mapping subunit is used to map the two-dimensional coordinate vector of each feature point in the feature point set to the camera coordinate system using the camera intrinsic parameters, to obtain the ray direction vector corresponding to each two-dimensional coordinate vector; to map each ray direction vector to the UAV body coordinate system using the camera extrinsic parameters, to obtain the three-dimensional coordinate vector corresponding to each ray direction vector; to map each three-dimensional coordinate vector to the world coordinate system using the flight attitude parameters of this UAV; to map each three-dimensional coordinate vector in the world coordinate system to the global coordinate system using the camera depth value, to obtain the global three-dimensional coordinate vector corresponding to each three-dimensional coordinate vector; and to store each global three-dimensional coordinate vector into a global three-dimensional coordinate vector set. The velocity extraction subunit is used to extract the global three-dimensional coordinate vector of any feature point at adjacent data acquisition times from the global three-dimensional coordinate vector set, and obtain the velocity vector of each feature point based on the extracted global three-dimensional coordinate vector. The vector generation subunit is used to combine the global 3D coordinate vector with the velocity vector to obtain the global observation vector for each feature point and establish a global observation vector set.

4. The system for reconstructing a 3D model of a moving object based on multi-UAV collaborative operation according to claim 3, characterized in that, The location acquisition module includes: The center point coordinate acquisition unit is used to obtain the center point coordinate vector of the target object at the next data acquisition moment based on the global observation vector set. The vertical feasible region acquisition unit is used to obtain the data based on the shooting angle constraint, relative distance constraint, and center point coordinate vector. z The axis component is used to obtain the vertical feasible region of this UAV at the next data acquisition moment; The horizontal feasible region acquisition unit is used to obtain the horizontal feasible region based on shooting angle constraints, relative distance constraints, and center point coordinate vectors. x Axis components and center point coordinate vectors y The axis component is used to obtain the horizontal feasible region of this UAV at the next data acquisition moment; The 3D feasible region generation unit is used to generate the 3D feasible region of the UAV at the next data acquisition moment based on the vertical feasible region and the horizontal feasible region. The target observation point selection unit is used to select the point closest to the location of the UAV at the current data acquisition time within the three-dimensional feasible domain as the target observation point.

5. A method for reconstructing a three-dimensional model of a moving object based on multi-unmanned aerial vehicle cooperation, characterized in that, Based on multiple drones, perform the following steps on each drone: Establish a two-way communication link between this drone and every other drone to form a mesh network; The system synchronously collects the flight status parameters of the UAV and the motion images of the target object, adds timestamps to the flight status parameters and motion images, and calls the self-organizing network communication module to broadcast the flight status parameters, motion images and corresponding timestamps through the mesh network. Based on the flight state parameters of the UAV at the current data acquisition time and the motion image of the target object at the current data acquisition time, the motion state of the target object at the next data acquisition time is predicted, and the global state vector set of the target object at the next data acquisition time is obtained. Based on the global state vector set, shooting angle constraints, relative distance constraints between the UAV and the target object, and the position of the UAV at the current data acquisition time, the optimal observation point of the UAV at the next data acquisition time is obtained; The drone is controlled to fly from its current location at the time of data acquisition to the target observation point at the next time of data acquisition. The three-dimensional model of the target object is reconstructed using motion images collected by this UAV at the corresponding target observation point and motion images collected by other UAVs at the corresponding target observation point; Obtaining the global state vector set of the target object at the next data acquisition moment includes the following steps: Based on the flight status parameters of this UAV at the current data acquisition time and the motion image of the target object at the current data acquisition time, generate a global observation vector for each feature point of the target object at the current data acquisition time, and establish a global observation vector set; Based on the global observation vector set, the motion state of each feature point of the target object at the next data acquisition moment is predicted, a state vector set is established, and the self-organizing network communication module is called to broadcast the state vector set of this UAV through the mesh network, and to receive the state vector set broadcast by each of the other UAVs through the mesh network. Based on the state vector set of this UAV and the received state vector set, the state vector set of this UAV is updated by weighted averaging to obtain the global state vector set.

6. The method of claim 5, wherein, Flight status parameters include: the real-time position, real-time flight speed, and real-time flight attitude of this UAV.

7. The method of claim 5, wherein, Establishing a global observation vector set includes the following steps: The target object in the motion image at the current data acquisition time is located by a deep learning object detection model, and a two-dimensional bounding box of the target object is generated. Multiple feature points of the target object are extracted from the sub-image corresponding to the two-dimensional bounding box using a feature extraction algorithm, resulting in a set of feature points. Using the camera intrinsic parameters, the two-dimensional coordinate vector of each feature point in the feature point set is mapped to the camera coordinate system to obtain the ray direction vector corresponding to each two-dimensional coordinate vector. Using the camera extrinsic parameters, each ray direction vector is mapped to the UAV body coordinate system to obtain the three-dimensional coordinate vector corresponding to each ray direction vector. Using the flight attitude parameters of this UAV, each three-dimensional coordinate vector is mapped to the world coordinate system. Using the camera depth value, each three-dimensional coordinate vector in the world coordinate system is mapped to the global coordinate system to obtain the global three-dimensional coordinate vector corresponding to each three-dimensional coordinate vector. Each global three-dimensional coordinate vector is stored in the global three-dimensional coordinate vector set. Extract the global three-dimensional coordinate vector of any feature point at adjacent data acquisition times from the global three-dimensional coordinate vector set, and obtain the velocity vector of each feature point based on the extracted global three-dimensional coordinate vector; By combining the global 3D coordinate vector with the velocity vector, the global observation vector of each feature point is obtained, and a global observation vector set is established.

8. The method of claim 7, wherein, To obtain the optimal observation point for this UAV at the next data acquisition time, the following steps are included: Obtain the center point coordinate vector of the target object at the next data acquisition moment based on the global observation vector set; Based on the shooting angle constraint, relative distance constraint, and center point coordinate vector z The axis component is used to obtain the vertical feasible region of this UAV at the next data acquisition moment; Based on shooting angle constraints, relative distance constraints, and center point coordinate vector x Axis components and center point coordinate vectors y The axis component is used to obtain the horizontal feasible region of this UAV at the next data acquisition moment; The three-dimensional feasible domain of the cost UAV at the next data acquisition moment is generated based on the vertical and horizontal feasible domains. Within the three-dimensional feasible domain, select the point closest to the location of the UAV at the current data acquisition time as the target observation point.

9. A computer device comprising a memory, a processor and a transceiver connected in communication sequence, wherein, The memory is used to store computer programs, the transceiver is used to send and receive data, and the processor is used to read the computer programs and execute the method for reconstructing a 3D model of a moving object based on multi-UAV collaborative operation as described in any one of claims 5-8. 10.A computer readable storage medium, having stored thereon instructions which, when executed on a computer, perform a method for reconstructing a three-dimensional model of a moving object based on multi-UAV cooperation according to any one of claims 5-8.

11. A computer program product containing instructions, which, when executed on a computer, cause the computer to perform a method for reconstructing a 3D model of a moving object based on multi-UAV cooperative operation as described in any one of claims 5-8; the computer comprising: General purpose computer, special purpose computer or programmable device.

Images