Multi-view data fusion and complementation system for sheltered area of unmanned aerial vehicle

Abstract

The invention discloses a multi-view data fusion and completion system for an unmanned aerial vehicle occlusion area, and relates to the technical field of unmanned aerial vehicle data processing. The system comprises a data acquisition module used for acquiring multi-view point cloud data and RGB images; the data preprocessing module is used for preprocessing the multi-view point cloud data and the RGB image to obtain fused point cloud data and a multi-view RGB optimized image; the three-dimensional model construction module is used for constructing a three-dimensional scene geometric model based on the fused point cloud data and analyzing to obtain a three-dimensional shielding area; the model repairing module is used for complementing the three-dimensional shielding area to obtain a non-shielding three-dimensional scene model; the texture completion module is used for carrying out texture fusion on the unshielded three-dimensional scene model based on the multi-view RGB optimized image to obtain a complete 3D scene model; and the virtual view angle output module is used for projecting the complete 3D scene model to generate a 2D view angle image. According to the invention, multi-view data fusion completion of the unmanned aerial vehicle shielding area is realized, and the completeness and accuracy of the three-dimensional scene model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of drone data processing, and in particular to a multi-view data fusion and completion system for drone occluded areas. Background Art

[0002] In the field of drone-mounted 3D scene data acquisition and processing, multi-view data acquisition using drone-mounted sensors has been widely used in scenarios such as terrain mapping and urban modeling. However, due to spatial occlusion by objects in the scene and the limited flight angle of the drone, the collected multi-view point cloud data and RGB images often contain occluded areas, resulting in missing geometric structure or incomplete texture information when constructing 3D scene models.

[0003] Therefore, how to efficiently fuse multi-view data to accurately identify and complete occluded areas and construct a complete and high-precision 3D scene model is a technical challenge that needs to be solved in the field of drone data processing. To this end, a multi-view data fusion and completion system for drone occluded areas is proposed. Summary of the Invention

[0004] The purpose of the present invention is to provide a multi-view data fusion and completion system for UAV occluded areas to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] A multi-view data fusion and completion system for UAV occluded areas, including a data acquisition module, a data preprocessing module, a 3D model construction module, a model repair module, a texture completion module, and a virtual view output module;

[0007] The data acquisition module is used to obtain multi-view point cloud data and multi-view RGB images of scenes with occluded areas at different viewing directions through a drone;

[0008] The data preprocessing module is used to perform point cloud denoising and point cloud registration on the multi-view point cloud data to obtain fused point cloud data, and simultaneously perform image denoising and image enhancement on the multi-view RGB image to obtain a multi-view RGB optimized image;

[0009] The three-dimensional model construction module is used to construct a three-dimensional scene geometric model based on the fused point cloud data, and perform visibility analysis on the three-dimensional scene geometric model to obtain a three-dimensional occlusion area of the three-dimensional scene geometric model;

[0010] The model repair module is used to complete the three-dimensional occluded area of the three-dimensional scene geometric model to obtain an unoccluded three-dimensional scene model.

[0011] The texture completion module is used to map the multi-view RGB optimized image texture to the unobstructed three-dimensional scene model, and perform texture fusion on the texture-mapped unobstructed three-dimensional scene model to obtain a complete 3D scene model;

[0012] The virtual perspective output module is used to project the complete 3D scene model to obtain 2D perspective images of the occluded area under different perspective directions.

[0013] Preferably, in the data acquisition module, the process of obtaining multi-view point cloud data and multi-view RGB images of a scene with occluded areas at different viewing angles by using a drone is as follows:

[0014] The UAV flight route is set through the UAV route planning software. The UAV flight route must meet the multi-angle coverage requirements of the occluded area. Specifically, the uniform distribution of spatial perspective is achieved by setting the waypoint spacing, flight altitude and attitude angle parameters. The UAV equipped with a 16-line lidar and a 4K resolution RGB camera is flown according to the preset route. During the flight, the UAV obtains real-time posture data through its own inertial measurement unit IMU and Beidou satellite navigation system. At the same time, the scenes with occluded areas are respectively obtained through the lidar and RGB camera. Set the acquisition frequency at different viewing angles Collect multi-view point cloud data and multi-view RGB images. The LiDAR simultaneously records the 3D coordinates of the point cloud, and the RGB camera simultaneously records the exposure parameters and color channel data of the image. All collected multi-view point cloud data and multi-view RGB images are accompanied by time stamps.

[0015] The acquisition frequency According to the flight speed of the drone Scene characteristic scale with occluded areas Set, satisfy the sampling theorem .

[0016] Preferably, in the data preprocessing module, the process of obtaining the fused point cloud data and the multi-view RGB optimized image is as follows:

[0017] The multi-view point cloud data is denoised by voxel grid filtering method, specifically using a side length of A cubic voxel grid, where The value range is 0.01m to 0.1m, and is based on the average point spacing of the point cloud. Dynamic adjustment to meet In order to preserve geometric features, downsampling is achieved by calculating the centroid coordinates of the point cloud within each voxel. The amount of point cloud data is reduced while removing outlier noise points, achieving point cloud denoising for multi-view point cloud data. The multi-view point cloud data after point cloud denoising is then registered using the iterative closest point algorithm. The 3D geometric feature descriptor of the point cloud is first calculated using the fast point feature histogram algorithm, and the initial pose estimation is achieved through feature matching. Then, iterative optimization is performed based on the iterative closest point algorithm. The termination conditions are that the root mean square error (RMSE) of the corresponding point pairs is less than 0.02m or the number of iterations reaches 50. The point clouds in the multi-view point cloud data from different perspectives are unified into the same coordinate system to obtain the fused point cloud data.

[0018] At the same time, the non-local mean filtering algorithm is used to denoise the multi-view RGB image, and the search window size is set to 21×21 pixels, the matching window size is 7×7 pixels, and the Gaussian standard deviation of the similarity weight calculation is A value of 10 suppresses image noise while preserving edge details through weighted averaging, achieving image denoising of multi-view RGB images. The denoised multi-view RGB images are then enhanced using adaptive histogram equalization. The images are divided into 8×8 pixel sub-blocks with a contrast limit of 40. Noise amplification is prevented by limiting the slope of the local histogram. After histogram equalization of each sub-block, bilinear interpolation is used to process block boundaries, resulting in a multi-view RGB optimized image with enhanced contrast and clear details.

[0019] The iterative closest point algorithm is a point cloud registration algorithm;

[0020] The non-local mean filtering algorithm is a denoising algorithm based on image block similarity.

[0021] Preferably, the fused point cloud data is processed by an octree-based point cloud fusion algorithm to obtain a three-dimensional scene geometric model:

[0022] Construct an octree space partitioning structure with a maximum depth of D, where the voxel space edge length corresponding to the leaf node is If the distance between the point cloud and the point cloud is greater than or equal to 0.05m and less than or equal to 0.5m, each point in the fused point cloud data is recursively inserted into the octree structure according to the three-dimensional coordinates, and the point cloud in each leaf node is weighted fused. During the weighted fusion process, a statistical outlier filtering strategy is used to calculate the distance from the point cloud in the leaf node to the centroid, and noise points with a value exceeding 2 times the standard deviation are removed. Based on this, the boundaries of the leaf nodes occupied by the point cloud in the octree are extracted to generate a three-dimensional scene geometric model;

[0023] The statistical outlier filtering strategy is a point cloud noise removal method based on the statistical characteristics of normal distribution.

[0024] Preferably, in the three-dimensional model construction module, the process of performing visibility analysis on the three-dimensional scene geometric model to obtain the three-dimensional occlusion area of the three-dimensional scene geometric model is as follows:

[0025] The geometric model of the 3D scene The first Voxel , and perspectives , defining visibility metrics , when voxel In perspective When visible 1, invisible is 0, based on which the geometric model of the three-dimensional scene is The comprehensive visibility score of each voxel is calculated using the comprehensive visibility score formula. , and the comprehensive visibility score corresponding to each voxel Passing visibility threshold Make judgments;

[0026] If the comprehensive visibility score Less than or equal to the visibility threshold , it is represented as an invisible voxel;

[0027] If the comprehensive visibility score Greater than visibility threshold , it is represented as a visible voxel;

[0028] The comprehensive visibility score formula is:

[0029] ;

[0030] in, is the weight factor;

[0031] The weighting factor According to the viewing angle Distance to voxel and the pitch angle of view The calculation formula is: , used to ensure greater visibility at close distances and vertical viewing angles;

[0032] Based on this statistics, the geometric model of the three-dimensional scene is obtained. All invisible voxels in the voxel are searched by breadth-first search algorithm. At the same time, according to the eight-neighborhood rule, the invisible voxels with connected relationships are traversed and marked to form connected regions. Until each invisible voxel in all the invisible voxels is divided into the corresponding connected region, the surface area of each connected region is calculated by the connected region surface area formula. , and the surface area of the connected region With surface area threshold Make judgments;

[0033] If the surface area of the connected region Less than the surface area threshold , then the connected area is not a real three-dimensional occlusion area;

[0034] If the surface area of the connected region Greater than or equal to the surface area threshold , then the connected area is a three-dimensional occlusion area;

[0035] Based on this, the three-dimensional occlusion area of the three-dimensional scene geometric model is obtained;

[0036] The formula for the surface area of the connected region is:

[0037] ;

[0038] in, are invisible voxels, is the voxel side length, is the face visibility function, These are the 6 fundamental planes of invisible voxels;

[0039] The surface visibility function is when the invisible voxel face When the adjacent invisible voxel does not belong to the current connected area, is 1, otherwise is 0;

[0040] The eight-neighborhood rule refers to six adjacent invisible voxels above, below, left, right, in front and behind the invisible voxel.

[0041] Preferably, the model repair module completes the three-dimensional occluded area of the three-dimensional scene geometric model by a completion algorithm based on deep learning to obtain an unoccluded three-dimensional scene model;

[0042] The completion algorithm based on deep learning is 3D CNN or Occupancy Networks.

[0043] Preferably, in the texture completion module, the process of mapping the multi-view RGB optimized image texture to the unobstructed three-dimensional scene model is as follows:

[0044] The coordinates of each image pixel in the multi-view RGB optimized image are calculated using the projection equation to obtain the coordinates of the model surface point corresponding to each image pixel coordinate in the unobstructed three-dimensional scene model. Based on this, the RGB value of the pixel corresponding to each image pixel coordinate is assigned to the unobstructed three-dimensional scene model surface point corresponding to the corresponding model surface point coordinate, thereby realizing the texture mapping of the multi-view RGB optimized image to the unobstructed three-dimensional scene model;

[0045] The projection equation is:

[0046] ;

[0047] in, are the image pixel coordinates, is the RGB camera internal parameter, is the RGB camera external parameter, and and In matrix form, are the coordinates of the model surface point.

[0048] Preferably, in the texture completion module, the process of performing texture fusion on the texture-mapped unobstructed three-dimensional scene model is as follows:

[0049] The texture-mapped unobstructed three-dimensional scene model is processed in sequence by an image interpolation algorithm and a Poisson fusion algorithm. The image interpolation algorithm uses a bicubic interpolation algorithm to perform smooth transition processing on the texture overlapping areas in the texture-mapped unobstructed three-dimensional scene model. The texture-mapped unobstructed three-dimensional scene model after smooth transition processing is subjected to a Poisson fusion algorithm based on a gradient domain to achieve texture global consistency optimization. Based on this, texture fusion is performed on the texture-mapped unobstructed three-dimensional scene model to obtain a complete 3D scene model.

[0050] Preferably, in the virtual perspective output module, the process of obtaining the 2D perspective image of the occluded area under different perspective directions is as follows:

[0051] Receive projection parameters and viewing angle parameters of the viewing direction input by the user through an API interface or a graphical user interface GUI. The viewing angle parameters support two representations: Euler angles or quaternions. The projection parameters are divided into orthogonal projection parameters including viewport size and projection range, and perspective projection parameters including field of view angle FOV, near clipping plane near, and far clipping plane far. Based on the input projection parameters and viewing angle parameters, perform orthogonal projection or perspective projection on the complete 3D scene model to generate 2D viewing angle images of the occluded area under different viewing angles.

[0052] The perspective projection is a projection method that simulates the principle of human vision;

[0053] The orthogonal projection is a projection method that directly maps an object in a three-dimensional space to a two-dimensional plane.

[0054] Due to the adoption of the above technical solution, the present invention has the following technical advancements compared to the prior art:

[0055] 1. This invention implements multi-view data fusion and intelligent geometric completion technology, significantly improving the integrity and accuracy of three-dimensional models. Through the coordinated acquisition of multi-view point clouds and RGB images from drones, combined with voxel grid filtering, iterative registration, and octree-based three-dimensional modeling, high-precision fusion of multi-source data is achieved. At the same time, comprehensive visibility scores and connected region analysis are used to accurately identify real occluded areas, and semantically aware completion is performed through a deep learning-based completion algorithm. This solves the problems of missing geometric structures and incorrect repairs in traditional methods, improves model integrity in complex scenes, and enhances the accuracy of occluded area completion.

[0056] 2. The present invention realizes gradient domain texture fusion and virtual perspective projection, achieving visual consistency and interactive scene analysis. It optimizes image textures from multiple perspectives by mapping projection equations, and adopts bicubic interpolation and Poisson fusion algorithms to eliminate texture splicing traces, making the visual effect more natural. At the same time, it supports virtual perspective output of orthogonal projection or perspective projection, breaking through the traditional fixed perspective limitation and providing multi-angle real-time analysis capabilities for scenes with occluded areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0058] Figure 1 It is a schematic diagram of the system function module flow of the present invention. DETAILED DESCRIPTION

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0060] Examples, such as Figure 1The multi-perspective data fusion and completion system for the occluded area of a drone includes a data acquisition module, a data preprocessing module, a three-dimensional model construction module, a model repair module, a texture completion module and a virtual perspective output module, which work together to complete the fusion and completion of the occluded area.

[0061] The data acquisition module is used to obtain multi-view point cloud data and multi-view RGB images of scenes with occluded areas at different viewing directions through a drone;

[0062] The data preprocessing module is used to perform point cloud denoising and point cloud registration on the multi-view point cloud data to obtain fused point cloud data, and simultaneously perform image denoising and image enhancement on the multi-view RGB image to obtain a multi-view RGB optimized image;

[0063] The three-dimensional model construction module is used to construct a three-dimensional scene geometric model based on the fused point cloud data, and perform visibility analysis on the three-dimensional scene geometric model to obtain a three-dimensional occlusion area of the three-dimensional scene geometric model;

[0064] The model repair module is used to complete the three-dimensional occluded area of the three-dimensional scene geometric model to obtain an unoccluded three-dimensional scene model.

[0065] The texture completion module is used to map the multi-view RGB optimized image texture to the unobstructed three-dimensional scene model, and perform texture fusion on the texture-mapped unobstructed three-dimensional scene model to obtain a complete 3D scene model;

[0066] The virtual perspective output module is used to project the complete 3D scene model to obtain 2D perspective images of the occluded area under different perspective directions.

[0067] Furthermore, the working principle of the present invention is described below by way of examples:

[0068] Assume that the 3D modeling of building facades blocked by trees in an urban complex is taken as an example.

[0069] The system uses a DJI M300 RTK drone equipped with a 16-line lidar and a 4K resolution RGB camera. The drone is equipped with an inertial measurement unit (IMU) and a Beidou satellite navigation system to obtain real-time pose data. The data processing terminal uses a workstation equipped with an NVIDIA RTX 3090 graphics card, 64GB of memory, and an Intel i9-11900K processor to meet the computing requirements of point cloud processing and deep learning models.

[0070] Use the drone route planning software DJI Terra to set a flight route around the building, with the waypoint spacing set to 5 meters, the flight altitude set to 40 meters, and the attitude angle parameters including pitch angle to , yaw angle to To ensure uniform coverage of the building facades blocked by trees from 8 different perspectives, the drone flies at a speed of 5m / s according to the preset route. According to the sampling theorem f≥2v / d, where the scene feature scale d is the average size of the gaps between tree branches and leaves, which is 0.5m, the acquisition frequency f is set to 20Hz, and multi-perspective point cloud data and multi-perspective RGB images with timestamps are acquired synchronously.

[0071] For the collected multi-view point cloud data, the side length The 0.05m cubic voxel grid is used for voxel grid filtering and denoising. Downsampling is achieved by calculating the voxel centroid, outlier noise points are removed, and point cloud denoising is achieved. The 3D geometric feature descriptor is calculated using the fast point feature histogram algorithm for the multi-view point cloud data after point cloud denoising. The initial pose estimation is achieved through feature matching, and then iterative optimization is performed based on the iterative closest point algorithm. The root mean square error of the corresponding point pairs is less than 0.02m as the termination condition. The point clouds of different viewpoints are unified into the same coordinate system to obtain fused point cloud data. For multi-view RGB images, a 21×21 pixel search window and a 7×7 pixel matching window are used, and the Gaussian standard deviation is 0. The non-local mean filtering algorithm with a value of 10 is used for denoising to preserve edge details, and the image is enhanced by the adaptive histogram equalization method. The image is divided into 8×8 pixel sub-blocks, the contrast limit is set to 40, and the block boundaries are processed by bilinear interpolation to obtain a contrast-enhanced multi-view RGB optimized image.

[0072] The fused point cloud data is input into the point cloud fusion algorithm based on octree, and an octree space partitioning structure with a maximum depth D of 10 is constructed, and the leaf node voxel side length is The point cloud is recursively inserted into the point cloud, and the point cloud in each leaf node is weighted fused. The statistical outlier filtering strategy is used to extract the leaf node boundary occupied by the point cloud, and a 3D scene geometric model is generated. The visibility analysis of N voxels in the 3D scene geometric model is performed, and the comprehensive visibility score formula is used. Calculate the comprehensive visibility score for each voxel, where the weight factor , set visibility threshold is 0.3, if , it is determined to be an invisible voxel. The invisible voxels are traversed by the breadth-first search algorithm according to the eight-neighborhood rule to form a connected region. The surface area formula of the connected region is used ,in The surface area of each connected region is calculated to set the surface area threshold. For 0.5m², The connected area is determined as the three-dimensional occluded area, such as the facade area of a building blocked by trees.

[0073] The 3D CNN deep learning algorithm is used to complete the three-dimensional occluded area. The geometric features of the adjacent visible voxels are used as input to predict the voxel occupancy value of the occluded area, generate an unoccluded three-dimensional scene model, perform texture mapping on the unoccluded three-dimensional scene model, and use the projection equation to calculate the occupancy value of the voxel in the occluded area. , the pixel coordinates of the multi-view RGB optimized image are mapped to the coordinates of the model surface points and assigned corresponding RGB values. Based on this, the texture-mapped unobstructed three-dimensional scene model is first smoothed by the bicubic interpolation algorithm in the texture overlapping area, and then the gradient domain-based Poisson fusion algorithm is used to optimize the global consistency of the texture, eliminate the splicing traces, and obtain a complete 3D scene model.

[0074] The user inputs the viewing angle parameters through the graphical user interface GUI, and the Euler angle is expressed as the pitch angle , yaw angle and roll angle And perspective projection parameters, perspective projection parameters are field of view FOV , near clipping plane near1m, far clipping plane far100m and viewport size 1920×1080 pixels, the virtual perspective output module performs perspective projection on the complete 3D scene model to generate a 2D perspective image of the obscured facade of the building, realizing multi-angle scene analysis.

[0075] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A multi-view data fusion and completion system for UAV occluded areas, characterized by: include: The data acquisition module is used to obtain multi-view point cloud data and multi-view RGB images of scenes with occluded areas in different viewing directions through drones; The data preprocessing module is used to perform point cloud denoising and point cloud registration on multi-view point cloud data to obtain fused point cloud data, and to perform image denoising and image enhancement on multi-view RGB images to obtain multi-view RGB optimized images; A 3D model construction module is used to construct a 3D scene geometric model based on the fused point cloud data, and perform visibility analysis on the 3D scene geometric model to obtain the 3D occlusion area of the 3D scene geometric model; The model repair module is used to complete the three-dimensional occluded area of the three-dimensional scene geometric model to obtain an unoccluded three-dimensional scene model; The texture completion module is used to map the multi-view RGB optimized image texture to the unobstructed 3D scene model, and perform texture fusion on the texture-mapped unobstructed 3D scene model to obtain a complete 3D scene model; The virtual perspective output module is used to project the complete 3D scene model to obtain 2D perspective images of the occluded area under different viewing directions.

2. The multi-view data fusion and completion system for UAV occluded areas according to claim 1 is characterized in that: In the data acquisition module, the process of obtaining multi-view point cloud data and multi-view RGB images of scenes with occluded areas at different viewing angles by using drones: The drone flight route is set by the drone route planning software, and the drone equipped with lidar and RGB camera is flown according to the drone flight route. At the same time, the scenes with blocked areas are respectively detected by lidar and RGB camera. Set the acquisition frequency at different viewing angles Collect multi-view point cloud data and multi-view RGB images.

3. The multi-view data fusion and completion system for UAV occluded areas according to claim 2, characterized in that: In the data preprocessing module, the process of obtaining the fused point cloud data and the multi-view RGB optimized image is as follows: The multi-view point cloud data is denoised by the voxel grid filtering method, and the multi-view point cloud data after point cloud denoising is performed by the iterative nearest point algorithm to obtain the fused point cloud data. At the same time, the multi-view RGB image is denoised by the non-local mean filtering algorithm, and the denoised multi-view RGB image is enhanced by the adaptive histogram equalization method to obtain the multi-view RGB optimized image.

4. The multi-view data fusion and completion system for UAV occluded areas according to claim 3 is characterized in that: In the 3D model construction module, the process of constructing a 3D scene geometric model is as follows: The fused point cloud data is processed by an octree-based point cloud fusion algorithm to obtain a three-dimensional scene geometric model.

5. The multi-view data fusion and completion system for UAV occluded areas according to claim 4 is characterized in that: In the 3D model construction module, the process of performing visibility analysis on the 3D scene geometric model to obtain the 3D occlusion area of the 3D scene geometric model: The geometric model of the 3D scene The comprehensive visibility score of each voxel is calculated using the comprehensive visibility score formula. , and the comprehensive visibility score corresponding to each voxel Passing visibility threshold Make judgments; If the comprehensive visibility score Less than or equal to the visibility threshold , it is represented as an invisible voxel; If the comprehensive visibility score Greater than visibility threshold , it is represented as a visible voxel; Based on this statistics, the geometric model of the three-dimensional scene is obtained. All invisible voxels in the voxel are searched by breadth-first search algorithm. At the same time, according to the eight-neighborhood rule, the invisible voxels with connected relationships are traversed and marked to form connected regions. Until each invisible voxel in all the invisible voxels is divided into the corresponding connected region, the surface area of each connected region is calculated by the connected region surface area formula. , and the surface area of the connected region With surface area threshold Make judgments; If the surface area of the connected region Less than the surface area threshold , then the connected area is not a real three-dimensional occlusion area; If the surface area of the connected region Greater than or equal to the surface area threshold , then the connected area is a three-dimensional occlusion area; Based on this, the three-dimensional occlusion area of the three-dimensional scene geometric model is obtained.

6. The multi-view data fusion and completion system for UAV occluded areas according to claim 5, characterized in that: The model repair module completes the three-dimensional occluded area of the three-dimensional scene geometric model using a completion algorithm based on deep learning to obtain an unoccluded three-dimensional scene model; The completion algorithm based on deep learning is 3D CNN or Occupancy Networks.

7. The multi-view data fusion and completion system for UAV occluded areas according to claim 6, characterized in that: In the texture completion module, the process of mapping the multi-view RGB optimized image texture to the unobstructed 3D scene model: The coordinates of each image pixel in the multi-view RGB optimized image are calculated by the projection equation to obtain the coordinates of the model surface point corresponding to each image pixel coordinate in the unobstructed three-dimensional scene model. Based on this, the RGB value of the pixel corresponding to each image pixel coordinate is assigned to the unobstructed three-dimensional scene model surface point corresponding to the corresponding model surface point coordinate, thereby realizing the texture mapping of the multi-view RGB optimized image to the unobstructed three-dimensional scene model.

8. The multi-view data fusion and completion system for UAV occluded areas according to claim 7, characterized in that: In the texture completion module, the process of texture fusion of the texture-mapped unobstructed 3D scene model is as follows: The texture-mapped unobstructed three-dimensional scene model is processed in sequence by an image interpolation algorithm and a Poisson fusion algorithm, and texture fusion is performed on the texture-mapped unobstructed three-dimensional scene model accordingly to obtain a complete 3D scene model.

9. The multi-view data fusion and completion system for UAV occluded areas according to claim 8, characterized in that: In the virtual perspective output module, the process of obtaining 2D perspective images of the occluded area under different perspective directions is as follows: According to the viewing direction and projection parameters specified by the user, the complete 3D scene model is orthogonally projected or perspectively projected to generate a 2D viewing image of the occluded area.

Images