Two-waveband image three-dimensional reconstruction method under optimal shooting interval
By using a dual-band camera on the drone platform for three-dimensional image reconstruction with optimal shooting intervals, combined with SIFT, MVS and ICP algorithms, the problems of low efficiency and insufficient detection energy in traditional three-dimensional measurement methods are solved, achieving efficient and accurate 3D reconstruction effect.
Patent Information
- Application Number
- CN202410926625.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-12
AI Technical Summary
In traditional three-dimensional morphology measurement methods, contact type has high accuracy but low efficiency, while non-contact active measurement lacks detection energy during long-distance measurement, making it difficult to achieve efficient three-dimensional reconstruction.
Using the three-dimensional reconstruction method of dual-band image with optimal shooting interval, the visible light camera and infrared camera are fixed on the drone platform, and the feature point matching is performed using SIFT algorithm, and the three-dimensional scene reconstruction is performed using MVS and ICP algorithms. , and fuse multispectral data through voxel grid to obtain three-dimensional scenes in small target areas.
On the premise of ensuring a certain accuracy, the efficiency of three-dimensional reconstruction is improved, and the limitations of single spectral information are overcome through multi-spectral data fusion, improving the accuracy and information richness of three-dimensional reconstruction.
Smart Images

Figure SMS_1 
Figure SMS_2 
Figure SMS_3
Abstract
Description
Technical Field
[0001] The invention belongs to the field of three-dimensional reconstruction, and in particular relates to a three-dimensional reconstruction method of dual-band images under optimal shooting interval. Background Art
[0002] Traditional 3D shape measurement methods can be mainly divided into contact and non-contact 3D shape measurement according to the acquisition method. Non-contact measurement can be divided into active measurement and passive measurement. The contact 3D measurement method has high accuracy, but the imaging efficiency is extremely low, while the non-contact active measurement has the problem of low detection energy for long-distance measurement. Summary of the invention
[0003] The object of the present invention is to provide a method for three-dimensional reconstruction of dual-band images under optimal shooting interval.
[0004] The technical solution to achieve the purpose of the present invention is: a method for three-dimensional reconstruction of dual-band images under optimal shooting interval, the steps of which are as follows:
[0005] Step 1: Fix a visible light camera and an infrared camera on the UAV platform, and calibrate the visible light camera and the infrared camera respectively to obtain the intrinsic parameters and distortion coefficients of the cameras;
[0006] Step 2: The drone flies over the small target area and takes photos of the small targets at equal intervals, using both the visible light camera and the infrared camera to take image pairs each time;
[0007] Step 3: Filter out images with complete small target areas from the captured images, correct the images with complete small target areas, use SIFT algorithm to match feature points for image pairs at different intervals, and use triangulation to estimate the distance of small targets;
[0008] Step 4, determining the optimal interval for drone flight shooting according to the scoring formula of the small target area distance calculation result;
[0009] Step 5: Select n1 infrared image frames and n2 visible light image frames containing the complete small target area from the images taken by the drone at the optimal intervals; use the MVS algorithm to reconstruct the three-dimensional scene of the obtained n1 infrared image frames and n2 visible light image frames respectively, and then fuse the two results to finally obtain the three-dimensional scene of the small target area.
[0010] Preferably, the visible light camera and the infrared camera are calibrated separately using Zhang Zhengyou's plane calibration method, wherein the different imaging characteristics of the infrared camera for the same object at different temperatures are utilized, and a zoned heated chessboard calibration plate is used to provide clear corner points for the infrared camera calibration.
[0011] Preferably, in step 2, the drone flies above the small target area, and the specific method for equidistantly photographing the small target is: control the drone to fly above the small target area, the camera shoots downward, the flight altitude of the drone is adjusted so that the small target area occupies about one-third of the image area, and the RTK positioning module is used to set equidistant shooting points, each shooting point shoots multiple frames of images, and the shooting points are on the same horizontal line, and the small target is moved to shoot until the small target disappears from the screen.
[0012] Preferably, step 3 selects images with complete small target areas from the captured images, corrects the images with complete small target areas, uses SIFT method to match feature points of the corrected images under different baselines, and selects small target areas to calculate small target distances using triangulation method.
[0013] Preferably, the specific method of determining the optimal interval for drone flight shooting in step 4 is:
[0014] Calculate the accuracy, stability and confidence of target distance under different baselines respectively;
[0015] The result scoring formula is calculated based on accuracy, stability and confidence, and the highest score is the optimal interval, that is, the best baseline distance for shooting.
[0016] Preferably, the accuracy formula is as follows:
[0017]
[0018] The stability formula is as follows:
[0019]
[0020] The confidence formula is as follows:
[0021]
[0022] Where D0 is the actual target distance, is the average value of the target distance, σ is the standard deviation of D, N is the total number of image sequences containing complete small targets, n is the number of image sequences after cross selection, A, C, and S are accuracy, stability, and confidence, respectively.
[0023] Preferably, the scoring formula is as follows:
[0024]
[0025] Preferably, in step 5, the MVS algorithm is used to reconstruct the three-dimensional scene of the obtained n1 frame infrared image and n2 frame visible light image respectively, and then the results of the two are merged to finally obtain the three-dimensional scene of the small target area. The specific method is:
[0026] Step 5.1: Select the infrared image frame n1 and the visible light image frame n2 under the best baseline, and use the MVS algorithm to calculate the three-dimensional point cloud of the scene for the two images respectively;
[0027] Step 5.2: For the reconstruction results of the two images, use the ICP algorithm to find the best correspondence between the two images, and then align and register them to obtain a point cloud model with the details of the two point clouds;
[0028] Step 5.3: Using the voxel grid fusion strategy, the registered point cloud data is divided into voxel grids, and the points in each voxel are fused.
[0029] Preferably, for the reconstruction results under the two images, the ICP algorithm is used to find the best correspondence between the two images, and then alignment and registration are performed to obtain a point cloud model with two point cloud details. The specific process is as follows:
[0030] Select the initial transformation matrix, using translation)t x ,t y ,t z The (and rotation (R) parameters represent mapping the infrared point cloud to the visible light point cloud coordinate system, and the transformation matrix T that constitutes the rough registration is:
[0031]
[0032] For each point p in the infrared point cloud i , find the nearest neighbor point q in the visible light point cloud i . Both satisfy q i =T·p i ;
[0033] The Euclidean distance is used to calculate the error metric between the points in the infrared point cloud and the points in the visible light point cloud. The specific calculation formula is:
[0034] E(T)=∑ i d(p i ,q i )
[0035] The weighted least squares method is used to calculate the weight of each point, and the new transformation matrix is obtained by optimizing the change parameters for precise registration. The specific transformation optimization formula is:
[0036] T new =argmin T ∑ i ω i ·d(p i ,T·p i ) 2
[0037] Update the transformation matrix to T1 = Tnew T, update the transformation matrix iteratively until convergence, and get the final transformation matrix T end , is the rigid transformation matrix between the two point clouds, that is, the point cloud precise registration transformation matrix is obtained, where ω i is the weight coefficient of the point.
[0038] Preferably, a voxel grid fusion strategy is adopted to segment the registered point cloud data into voxel grids, and the specific process of fusing the points in each voxel is as follows:
[0039] Solve the average value of all voxel points in the point cloud to generate a fused point cloud model;
[0040] The fusion weight is adjusted according to the infrared point cloud information of the small target area. The point cloud after weight adjustment is the final point cloud fusion result, that is, the 3D reconstruction of the small target area with richer information is completed.
[0041] Compared with the prior art, the present invention has the following significant advantages: the present invention adopts the optimal shooting interval technology to improve the efficiency of 3D reconstruction while ensuring a certain reconstruction accuracy; the dual-band 3D reconstruction technology overcomes the information limitations of a single spectrum through multi-spectral data fusion, thereby displaying more comprehensive and rich scene information in a single point cloud model, thereby improving the accuracy of 3D reconstruction.
[0042] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 This is the system architecture diagram.
[0044] Figure 2 Schematic diagram of the infrared calibration board.
[0045] Figure 3 Select a schematic for Interval.
[0046] Figure 4 Schematic diagram of 3D voxel grid fusion. DETAILED DESCRIPTION
[0047] A method for 3D reconstruction of dual-band images at optimal shooting intervals. First, a visible light camera and an infrared camera are fixed on a UAV platform, and calibration is performed to obtain the camera internal parameters and distortion coefficients. The UAV flies above the small target area and uses the RTK positioning module to accurately shoot. Select images with a complete small target area, use the SIFT method to match feature points for image pairs under different baselines (i.e., different intervals), and use the triangulation method to estimate the distance of the small target. After obtaining the target distance, the accuracy, stability, and confidence of the target distance under different baselines are calculated respectively. The highest score of the product of the three is the optimal interval, that is, the best baseline distance for shooting. Select the infrared image n1 frame and the visible light image n2 frame at the optimal interval, use the MVS algorithm to perform 3D reconstruction on the infrared image and the visible light image respectively, align the two reconstruction results with the ICP algorithm, and then use the voxel grid fusion strategy to fuse the two reconstruction results, and finally obtain the 3D scene of the small target area.
[0048] The present invention proposes an optimal interval selection technology for flight shooting, which optimizes the selection of stereo matching image pairs to the greatest extent, reduces repeated operations on each pair of video frames, and can accurately and quickly obtain three-dimensional data; it proposes a dual-band point cloud fusion technology, which displays richer scene information in a single point cloud model by adjusting the weight of the fusion of two bands of point clouds.
[0049] Step 1: Fix the visible light camera and infrared camera on the UAV platform, and calibrate the visible light camera and infrared camera respectively to obtain the camera's intrinsic parameters and distortion coefficients. The camera calibration adopts Zhang Zhengyou's plane calibration method, which uses the different imaging characteristics of the infrared camera for the same object at different temperatures, and uses a zoned heated chessboard calibration plate to provide clear corner points for infrared camera calibration. Figure 2 shown.
[0050] Step 2: Shoot the small target area. Control the drone to fly above the small target area, adjust the camera to shoot downward, adjust the drone's flight altitude so that the small target area occupies about one-third of the image area, and use RTK to set a small interval to move horizontally to shoot the small target until the small target disappears from the screen.
[0051] Step 3, small target distance calculation, select the image containing the complete small target area according to the time axis of the drone shooting, perform distortion correction and epipolar correction on the image, use the SIFT (Scale Invariant Feature Transform) algorithm to match the feature points of the corrected images at different intervals, and select the small target area to calculate the small target distance using the triangulation method.
[0052] Step 4: Determine the optimal shooting interval. Calculate the accuracy, stability and confidence of the target distance under different baselines (i.e., different intervals); the accuracy formula is as follows:
[0053]
[0054] The stability formula is as follows:
[0055]
[0056] The confidence formula is as follows:
[0057]
[0058] Where D0 is the actual target distance, is the average value of the target distance, σ is The standard deviation of , N is the total number of image sequences containing complete small targets, n is the number of image sequences after cross selection, A, C, and S are accuracy, stability, and confidence, respectively.
[0059] The product of accuracy, stability and confidence is the calculation result scoring formula; the highest score is the optimal interval, that is, the best baseline distance for shooting. The scoring formula is as follows:
[0060]
[0061] The following is a detailed analysis of the impact of the selected interval and jump interval on the calculation results of the above formula.
[0062] Assume that each shooting point captures x images containing a complete small target, and let the selection interval a be the selection interval of the original image sequence. The jump interval b is used to select images in each point image sequence selected under the selection interval a. The smaller the selection interval a, the smaller the baseline, and the size of the jump interval b does not affect the size of the baseline between image pairs. When the selection interval a is larger, the number of selected image pairs decreases, and the relationship is linear when the jump interval remains unchanged; in addition, when the selection interval remains unchanged, the number of selected image pairs decreases to 1 / k of the original number for every increase of the jump interval to k times, which is inversely proportional, so there is the following relationship:
[0063]
[0064] In the above formula, n is only taken as integer divisible value, and the range of a is The value range of b is 1≤b≤x-1. The increase of jump interval will lead to a decrease in the number of selected images and lower confidence. The only advantage is that it can reduce the amount of calculation. Compared with confidence, reducing the amount of calculation is more important. When the selection interval is fixed, the image pairs selected by different jump intervals may be repeated. Specifically, when the jump interval is a multiple relationship, that is, when the latter jump interval is an integer multiple of the previous jump interval, the previous selected image pair set will include the latter image pair set. In this method, the jump interval as a control variable is of little significance. Ideally, the optimal jump interval is 1, that is, any two images with equal baselines in this interval will be selected in turn. Interval selection is as follows: Figure 3 As shown in , the first row shows all image sequences, where orange is the image in the selected interval, and the second row shows all images in the selected interval, where red is the image in the jump interval selection.
[0065] The jump interval b in this experiment is set to 1. The experiment sets the experimental results under the same target distance with the varying selection interval a and compares the results under different target distances under the same selection interval a. It can be seen that in the distance calculation of long-distance targets, the minimum baseline solution represented by the interval of 1 has a large error, and the calculated data is not stable enough. We gradually increase the selection interval and continue to calculate the target distance through two frames of images to obtain the target distance calculation results for the same target with the varying selection interval.
[0066] The experimental data obtained by experimental calculation with intervals of 1 to 15 when the target distance is about 1005m. As the interval increases, the data points of the polyline are fewer, corresponding to the reduction of data volume; in addition, the polyline will become more and more gentle, which represents the improvement of data stability. Next, we change the selected target. The actual values of the distance reference of the three selected targets are 1005.54m, 1493.53m and 1998.58m respectively. This is a more accurate estimate by solving the Euclidean distance by longitude, latitude and altitude. Repeat the same experimental steps to obtain experimental data that changes with the selected interval. Keep the average value of the above target data set for different selected intervals, and calculate its standard deviation data. The overall change curve of each group of experimental data can be obtained by sorting. When the selected interval is small, the calculated value is much different from the true value, reflecting the insufficient accuracy of the experimental data. This is because when shooting, there must be a certain parallax change for long-distance ranging, and the small baseline represented by the small interval may not obtain enough parallax to estimate the depth, resulting in insufficient accuracy of the calculated data. When the selected interval increases to a certain number, the average value of each group of experiments tends to be stable, with a small error and high precision. This shows that the baseline has increased to the point where the parallax required for long-distance ranging can be obtained, and more accurate long-distance ranging can be performed.
[0067] The average values under different intervals generally tend to the real distance. As the interval increases, the estimated target distance will be closer to the real value and the accuracy will be higher. This is because a larger baseline can provide a larger disparity range, which means that the depth perception of distant objects is more accurate. A larger distance between cameras will produce a larger disparity, which makes the depth calculation range wider. In addition, the larger distance between cameras makes the system less affected by some external factors (such as illumination changes, noise, etc.), which has certain advantages for applications in unstable environments. When the selected interval is small, the standard deviation of the experimental data is large, reflecting the instability of the data. This is because when the interval is small, the baseline of adjacent shooting positions is small, and the disparity of the image obtained by shooting with a too small baseline is too small, even reaching the pixel level. The disparity values extracted between different image pairs will have large fluctuations, which may cause the calculated experimental values to fluctuate more significantly. When the selected interval increases to a certain number, the disparity values between different image pairs are large and tend to be stable, so the experimental values calculated by the experiment are also relatively stable.
[0068] The curve changes of the mean and variance indicate that the accuracy and stability will tend to stabilize as the selection interval increases. Under a small selection interval, the accuracy and stability are not good, and better accuracy and stability can only be obtained when the interval increases to a certain value. In the case of a small baseline, the standard deviation of the data is large and the data is relatively unstable. This is because the small baseline limits the parallax range that the camera system can observe. A smaller baseline causes the camera to produce smaller disparity values, so for distant objects, the disparity values may be too small to effectively measure their depth, and mismatches in the disparity map may be more significant. This limits the depth perception ability of a small baseline camera system at long distances.
[0069] In the curves of different targets, the minimum intervals that can be achieved are different, that is, the minimum baselines required to achieve accuracy and stability are different. This shows that there is a minimum baseline for different targets, and the cross-selection optimization scheme under this baseline scheme can obtain better data accuracy and stability.
[0070] In addition, the number of experimental data will decrease linearly with the increase of the selection interval. If the number of data is too small, the confidence is poor and cannot well represent the statistical requirements of the experimental data. Therefore, for the overall evaluation of the cross-selection scheme, a theoretical basis for confidence is also required. Here, the ratio of the number of selected image pairs to the number of images in the image sequence is used as the confidence index. Since the selection interval changes linearly, the confidence index also changes linearly.
[0071] The mean value, standard deviation and the number of selected image pairs are used together as the evaluation basis for the cross-selection scheme. In the overall scheme evaluation, the formula (6) in step 4 is used The evaluation scores of different selected schemes with different objectives are calculated.
[0072] From the evaluation scores obtained, it can be seen that no matter what the distance of the target is, for the selection interval that is too small or too large, their evaluation scores are not high. The small baseline is due to the low accuracy and stability evaluation, and the large baseline is due to the low confidence evaluation. Even when the selection interval is equal to 15, the evaluation score is equal to 0. This is because only the first and last two frames of images are selected at this time, and only one experimental data is obtained. The standard deviation cannot be calculated, and it cannot meet the basic requirements of the experiment. The peak of the evaluation curve is generally in the middle of the selection interval. At this time, the baseline length represented by the selection scheme can not only meet the accuracy requirements of reconstruction, but also have a certain amount of data as support.
[0073] In addition, for targets at different distances, the selection intervals corresponding to the peaks of the score curves are also different. This shows that when shooting targets at different distances, the more suitable shooting baselines are also different. Moreover, the optimal selection interval will increase with the increase of the target distance. It can be understood that shooting targets at longer distances requires a larger baseline to obtain more obvious parallax changes. If this evaluation function is used to evaluate the selection scheme, the selection interval represented by the extreme point is the optimal selection interval for this image sequence. This selection interval is used to cross-select the original sequence, and the parallax calculation is performed to obtain a more accurate target measurement distance, and there is a certain amount of data to meet the statistical confidence requirements. By multiplying the optimal spacing by the minimum shooting spacing during actual shooting, a baseline length that is more suitable for three-dimensional measurement that meets the accuracy, stability and confidence can be obtained. This can help obtain image sequences. If the distance of the target to be shot is known, we can perform continuous shooting based on the optimal baseline calculated by this method, thereby reducing the size of the image data set within a reasonable range of accuracy and reducing the time of image solution, thereby improving the efficiency of three-dimensional reconstruction.
[0074] Step 5: Select n1 infrared image frames and n2 visible light image frames containing the complete small target area from the images taken by the drone at the optimal interval; use the MVS (Multi-View Stereo Reconstruction) algorithm to reconstruct the three-dimensional scene of the obtained n1 infrared image frames and n2 visible light image frames, and then fuse the two results to finally obtain the three-dimensional scene of the small target area. The specific method is:
[0075] Step 5.1: Select the infrared image frame n1 and the visible light image frame n2 under the best baseline, and use the MVS algorithm to calculate the three-dimensional point cloud of the scene for the two images respectively;
[0076] Step 5.2: For the reconstruction results of the two images, use the ICP algorithm to find the best correspondence between the two images, and then align and register them to obtain a point cloud model with the details of the two point clouds. The specific process of point cloud alignment and registration is as follows:
[0077] First, choose the initial transformation matrix, using the translation (t x ,t y ,t z ) and rotation (R) parameters represent mapping the infrared point cloud to the visible light point cloud coordinate system, and the transformation matrix T for coarse registration is:
[0078]
[0079] For each point p in the infrared point cloud i , we need to find the nearest neighbor point q in the visible light point cloud i . Both satisfy q i =T·p i .
[0080] The Euclidean distance is used to calculate the error metric between the points in the infrared point cloud and the points in the visible light point cloud. The specific calculation formula is:
[0081] E(T)=∑ i d(p i ,q i ) (8)
[0082] Finally, the weighted least squares method is used to calculate the weight of each point, and the new transformation matrix is obtained by optimizing the change parameters for precise registration. The specific transformation optimization formula is:
[0083] T new =argmin T ∑ i ω i ·d(p i ,T·p i ) 2 (9)
[0084] Update the transformation matrix to T1 = T new T, update the transformation matrix iteratively until convergence, and get the final transformation matrix T end , is the rigid transformation matrix between the two point clouds, that is, the point cloud precise registration transformation matrix is obtained, where ω i is the weight coefficient of the point;
[0085] Step 5.3: After obtaining the point cloud registration result, the voxel grid fusion strategy is adopted, such as Figure 4As shown in the figure, the registered point cloud data is divided into voxel grids, and the points in each voxel are fused. First, the average value of all points in the voxels in the point cloud is solved to generate the fused point cloud model. This step effectively merges the infrared point cloud and the visible light point cloud into a whole while retaining the detailed features of both.
[0086] Then adjust the fusion weight according to the infrared point cloud information of the small target area. The area with high temperature in the infrared point cloud is brighter and is an important part of the whole; while the area with low temperature is darker and closer to black, and the point cloud in this area is a non-important part of the infrared point cloud as a whole. Therefore, when fusing two registered point clouds of the same voxel grid, the weight will be adaptively adjusted according to the actual brightness of the infrared point cloud in the small target area. The brighter infrared point cloud has a higher weight, and the fusion result highlights the sensitivity of the point cloud to temperature; the darker infrared point cloud has a very low weight, and the fusion in this case is mainly based on the visible light point cloud with more information; the point cloud after adjusting the weight is the final point cloud fusion result, that is, the 3D reconstruction of the small target area with richer information is completed.
Claims
1. A dual-band image 3D reconstruction method at an optimal shooting interval, characterized in that: The steps to achieve this are as follows: Step 1: Fix a visible light camera and an infrared camera on the UAV platform, and calibrate the visible light camera and the infrared camera respectively to obtain the intrinsic parameters and distortion coefficients of the cameras; Step 2: The drone flies over the small target area and takes photos of the small targets at equal intervals, using both the visible light camera and the infrared camera to take image pairs each time; Step 3: Filter out images with complete small target areas from the captured images, correct the images with complete small target areas, use SIFT algorithm to match feature points for image pairs at different intervals, and use triangulation to estimate the distance of small targets; Step 4, determining the optimal interval for drone flight shooting according to the scoring formula of the small target area distance calculation result; Step 5: Select n1 infrared image frames and n2 visible light image frames containing the complete small target area from the images taken by the drone at the optimal intervals; use the MVS algorithm to reconstruct the three-dimensional scene of the obtained n1 infrared image frames and n2 visible light image frames respectively, and then fuse the two results to finally obtain the three-dimensional scene of the small target area.
2. The method for 3D reconstruction of dual-band images at optimal shooting interval according to claim 1, characterized in that: The visible light camera and infrared camera are calibrated separately using Zhang Zhengyou's plane calibration method. The different imaging characteristics of the infrared camera for the same object at different temperatures are utilized, and a zoned heated chessboard calibration plate is used to provide clear corner points for the infrared camera calibration.
3. The method for 3D reconstruction of dual-band images at optimal shooting interval according to claim 1, characterized in that: In step 2, the drone flies above the small target area, and the specific method for equidistantly photographing the small target is as follows: control the drone to fly above the small target area, shoot downward with the camera, adjust the drone's flight altitude so that the small target area occupies about one-third of the image area, use the RTK positioning module to set equidistant shooting points, capture multiple frames of images at each shooting point, and the shooting points are on the same horizontal line, and move the small target to shoot until the small target disappears from the screen.
4. The method for 3D reconstruction of dual-band images at optimal shooting interval according to claim 1, characterized in that: Step 3: Filter out images with complete small target areas from the captured images, correct the images with complete small target areas, use the SIFT method to match feature points of the corrected images under different baselines, and select the small target area to calculate the small target distance using the triangulation method.
5. The method for 3D reconstruction of dual-band images at optimal shooting interval according to claim 1, characterized in that: Step 4 The specific method for determining the optimal interval for drone flight photography is: Calculate the accuracy, stability and confidence of target distance under different baselines respectively; The result scoring formula is calculated based on accuracy, stability and confidence, and the highest score is the optimal interval, that is, the best baseline distance for shooting.
6. The method for 3D reconstruction of dual-band images at optimal shooting interval according to claim 5, characterized in that: The accuracy formula is as follows: The stability formula is as follows: The confidence formula is as follows: Where D0 is the actual target distance, is the average value of the target distance, σ is The standard deviation of , N is the total number of image sequences containing complete small targets, n is the number of image sequences after cross selection, A, C, and S are accuracy, stability, and confidence, respectively.
7. The method for 3D reconstruction of dual-band images at optimal shooting interval according to claim 6, characterized in that: The scoring formula is as follows:
8. The method for 3D reconstruction of dual-band images at optimal shooting interval according to claim 1, characterized in that: Step 5: Use the MVS algorithm to reconstruct the three-dimensional scene of the obtained n1 frame infrared image and n2 frame visible light image respectively, and then fuse the two results to finally obtain the three-dimensional scene of the small target area. The specific method is: Step 5.1: Select the infrared image frame n1 and the visible light image frame n2 under the best baseline, and use the MVS algorithm to calculate the three-dimensional point cloud of the scene for the two images respectively; Step 5.2: For the reconstruction results of the two images, use the ICP algorithm to find the best correspondence between the two images, and then align and register them to obtain a point cloud model with the details of the two point clouds; Step 5.3: Using the voxel grid fusion strategy, the registered point cloud data is divided into voxel grids, and the points in each voxel are fused.
9. The method for 3D reconstruction of dual-band images at optimal shooting interval according to claim 8, characterized in that: For the reconstruction results under the two images, the ICP algorithm is used to find the best correspondence between the two images, and then alignment and registration are performed to obtain a point cloud model with two point cloud details. The specific process is as follows: Select the initial transformation matrix, using translation (t x ,t y ,t z ) and rotation (R) parameters represent mapping the infrared point cloud to the visible light point cloud coordinate system, and the transformation matrix T for coarse registration is: For each point p in the infrared point cloud i , find the nearest neighbor point q in the visible light point cloud i . Both satisfy q i =T·p i ; The Euclidean distance is used to calculate the error metric between the points in the infrared point cloud and the points in the visible light point cloud. The specific calculation formula is: E(T)=∑ i d(p i ,q i ) The weighted least squares method is used to calculate the weight of each point, and the new transformation matrix is obtained by optimizing the change parameters for precise registration. The specific transformation optimization formula is: T new =argmin T ∑ i ω i ·d(p i ,T·p i ) 2 Update the transformation matrix to T1 = T new T, update the transformation matrix iteratively until convergence, and get the final transformation matrix T end , is the rigid transformation matrix between the two point clouds, that is, the point cloud precise registration transformation matrix is obtained, where ω i is the weight coefficient of the point.
10. The method for 3D reconstruction of dual-band images at optimal shooting interval according to claim 8, characterized in that: The strategy of voxel grid fusion is adopted to segment the registered point cloud data into voxel grids. The specific process of fusing the points in each voxel is as follows: Solve the average value of all voxel points in the point cloud to generate a fused point cloud model; The fusion weight is adjusted according to the infrared point cloud information of the small target area. The point cloud after weight adjustment is the final point cloud fusion result, that is, the 3D reconstruction of the small target area with richer information is completed.
Citation Information
Cited By
Shooting data management method and system of unmanned aerial vehicle based on intelligent processing
CN120091228A
Method and system for managing shooting data of unmanned aerial vehicle based on intelligent processing
CN120091228B
