Multispectral point cloud generation method based on multispectral image reconstruction, storage medium and equipment
By using a multispectral image reconstruction method, the problem of insufficient utilization of spectral information in existing technologies is solved, and high-precision and efficient multispectral point cloud generation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HARBIN INST OF TECH
- Filing Date
- 2023-09-13
- Publication Date
- 2026-06-26
AI Technical Summary
Existing image-based 3D reconstruction methods fail to fully utilize the spectral information of multispectral images, resulting in low reconstruction accuracy and efficiency.
Multispectral image reconstruction methods are employed, including radiometric correction and band alignment, feature extraction combining image enhancement and SIFT feature operators, NDVI mask matching and geometric verification, motion structure restoration, and multi-view geometric matching techniques, to generate multispectral point clouds.
It improves the accuracy and efficiency of 3D reconstruction of multispectral images, accurately recovers multispectral features and matching relationships, and generates high-precision multispectral point clouds.
Smart Images
Figure CN117218315B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of multispectral image three-dimensional reconstruction technology, specifically relating to a multispectral point cloud generation method, storage medium, and device. Background Technology
[0002] With the continuous development of 3D reconstruction technology, using UAVs to acquire RGB images for 3D reconstruction has become a hot technology for obtaining 3D scene information such as cities and forests. Multispectral sensors can acquire multispectral images of scenes, containing richer spectral information, which is of greater value for applications in agriculture and forestry. Existing image-based 3D reconstruction methods are designed for optical RGB images and do not consider the characteristics of multispectral images. When directly applied to multispectral images, they suffer from problems such as insufficient utilization of spectral information, low reconstruction accuracy, and low efficiency. Summary of the Invention
[0003] This invention aims to address the problems of insufficient utilization of spectral information, low reconstruction accuracy, and low efficiency in existing technologies.
[0004] A method for generating multispectral point clouds based on multispectral image reconstruction includes the following steps:
[0005] Step 1: Based on the multispectral camera and multispectral image acquisition parameters, perform radiometric correction and band alignment on the multispectral images; crop the overlapping areas of the multispectral images, and superimpose multiple single-band images to synthesize a band-aligned multispectral image.
[0006] Step 2: Obtain band-by-band features by combining image enhancement with the SIFT feature operator. After all band features have been extracted, fuse the multispectral features using the following method:
[0007] For any multispectral image, iterate through the position of each pixel in the image. If there is only one multispectral feature at the pixel position, retain that feature as the fused multispectral feature. If there are multiple multispectral features at the pixel position, filter these features and retain the feature with the largest feature amplitude as the multispectral fusion feature.
[0008] Step 3: Based on the multispectral image obtained in Step 1, calculate the NDVI value of the multispectral image and generate an NDVI mask; use the NDVI mask to match and geometrically verify the multispectral features fused in Step 2.
[0009] Step 4: Based on the feature matching pairs obtained in Step 3, establish the matching relationship of multispectral images in the image set, calculate the relative pose information of the images using the motion structure recovery method, calculate the image depth value using multi-view geometric matching technology, project the multispectral values, and generate a multispectral point cloud.
[0010] Furthermore, the radiation correction process in step 1 includes the following steps:
[0011] Based on the imaging principle of multispectral sensors, the mathematical model for radiometric correction of instantaneously acquired multispectral images is as follows:
[0012] L=g·V(x,y)·R(y)
[0013] Where L is the radiometric correction model for the multispectral image, g is the sensor gain coefficient, V(x,y) is the removal of edge light reduction effect caused by sensor lens, and R(y) is the row gradient correction factor.
[0014] The edge light reduction effect V(x,y) caused by removing the sensor lens is as follows:
[0015]
[0016] k = 1 + k0r + k1r 2 +k2r 3 +k3r 4 +k4r 5 +k5r 6
[0017]
[0018] Where k is the vignette correction factor, k α The coefficients of the correction polynomial are obtained from the sensor, α = 0, 1, ... 5, (c x ,c y (This is the sensor vignette correction center;)
[0019] The row gradient correction factor R(y) is as follows:
[0020]
[0021] Where I(x,y) is the original intensity value of pixel (x,y) in the image, I BL t represents the black level value of the image. e denoted as the exposure time of the multispectral image, and a1 and a2 as radiometric correction coefficients.
[0022] Furthermore, the band alignment process in step 1 includes the following steps:
[0023] S1.2.1 Perform geometric distortion correction on band-by-band images:
[0024] x'=x(1+j1r 2 +j2r 4 +j3r 6 )
[0025] y'=y(1+j1r 2 +j2r 4 +j3r 6 )
[0026] Where (x', y') represents the corrected pixel position, j β β represents the lens distortion correction factor, where β = 1, 2, 3;
[0027] S1.2.2 Perform geometric registration transformation on images of each band using sensor lens structure parameters:
[0028] Based on the positional differences between the lenses and the Euler angles, the Euler angles are converted into a rotation matrix using the following formula:
[0029]
[0030]
[0031]
[0032]
[0033] in, The three Euler angles between the shots;
[0034] By calculating the rotation matrix between lenses, geometric transformations can be performed on images of each band, enabling the registration and alignment of images acquired by multiple lenses.
[0035] Furthermore, the band-by-band features obtained by combining image enhancement with the SIFT feature operator are as follows:
[0036]
[0037]
[0038] Among them, I k For the k-th band of the multispectral image obtained in step one, CLAHE(·) represents the adaptive histogram equalization CLAHE algorithm processing. F represents the result of image enhancement. k The features extracted for the k-th band.
[0039] Furthermore, the process of calculating the NDVI value of the multispectral image and generating an NDVI mask based on the multispectral image obtained in step 1 includes the following steps:
[0040] Based on the multispectral image obtained in step 1, the NDVI value of the multispectral image is calculated according to the following formula, and an NDVI mask is generated:
[0041]
[0042]
[0043] Among them, I NDVI The NDVI value representing a multispectral image, I NIR For the near-infrared band of multispectral images, I R The red band value of the multispectral image; Mask V For NDVI masking, the value is 1 for vegetated areas and 0 for non-vegetated areas.
[0044] Furthermore, the process of matching and geometrically verifying the multispectral features fused in step 2 using an NDVI mask includes the following steps:
[0045] Based on the location of the NDVI mask and the fused multispectral features, the fused multispectral features are divided into F regions of vegetation areas. V Non-vegetated area F NV For these two features, the nearest neighbor algorithm is used for matching. During the feature matching process, the vegetation region feature F in the reference image is used. V Vegetation region features in other multispectral images only F V Matching is performed using the non-vegetated region features F in the reference image. NV Features of non-vegetated areas in other multispectral images only F NV Nearest neighbor matching is performed; and geometric verification is performed using random sample consistency to obtain feature matching pairs between multispectral images.
[0046] Furthermore, step 4 specifically includes the following steps:
[0047] Based on the feature matching pairs obtained in step 3, the matching relationship of multispectral images in the image set is established. The relative pose relationship between multispectral images is obtained according to the triangulation principle. The motion structure recovery method is used to perform three-dimensional reconstruction of the multispectral images, obtaining the three-dimensional relationship of the reconstructed feature points and the optimized image pose relationship based on optimization criteria. The optimization criteria are as follows:
[0048]
[0049] Among them, M i X represents the projection matrix of the i-th camera. j Let x be the reconstructed coordinates of the j-th 3D reconstruction point. ij is the image coordinate of the j-th 3D reconstruction point in the i-th image; E(M,X) is the loss function optimized by the bundle adjustment method, minimizing it can minimize the reprojection error;
[0050] The multispectral image and the optimized pose relationship are used as input for multi-view stereo vision to estimate the depth map of the reconstructed image. The spatial position of the reconstructed point cloud is obtained by using the pose relationship of the image and the depth map of the multispectral image. The multispectral values are then assigned to the corresponding spatial points through projection, and finally, the multispectral point cloud MSPC is generated.
[0051] Furthermore, the aforementioned multispectral point cloud MSPC is as follows:
[0052]
[0053] Among them, (x i ,y i ,z i () represents the three-dimensional coordinates of the i-th point. p is the spectral value of the nth band at the i-th point. num It represents the total number of points in the multispectral point cloud.
[0054] A computer storage medium storing at least one instruction, which is loaded and executed by a processor to implement the multispectral point cloud generation method based on multispectral image reconstruction.
[0055] A multispectral point cloud generation device based on multispectral image reconstruction is provided. The device includes a processor and a memory. The memory stores at least one instruction, which is loaded and executed by the processor to implement the multispectral point cloud generation method based on multispectral image reconstruction.
[0056] Beneficial effects:
[0057] This invention proposes a multispectral point cloud generation method based on 3D reconstruction of multispectral images. It accurately obtains multispectral features and their matching relationships, thereby recovering the 3D information of the multispectral image. This invention fully utilizes spectral information, effectively improving reconstruction accuracy. Furthermore, the method exhibits good reconstruction efficiency. To verify the performance of the proposed algorithm, experiments were conducted on multispectral images acquired by a UAV. The experimental results validate the effectiveness, accuracy, and efficiency of the proposed method. Attached Figure Description
[0058] Figure 1 Flowchart for generating multispectral point clouds.
[0059] Figure 2 The generated effect image of multispectral point cloud. Detailed Implementation
[0060] To address this, this invention designs a multispectral point cloud generation method based on multispectral image reconstruction. Taking into account the characteristics of multispectral sensors and multispectral images, radiometric correction and band alignment steps for multispectral images are added to the 3D reconstruction process. Furthermore, a fused multispectral feature extraction method is used to fully utilize the spectral information of the multispectral image, providing more potential reconstructed feature points. To eliminate mismatches caused by more feature points, an NDVI-guided multispectral feature matching method is designed to obtain more accurate matching relationships and improve the accuracy of multispectral 3D reconstruction.
[0061] Specific implementation method one: Combining Figure 1 This implementation method is described below.
[0062] This embodiment presents a multispectral point cloud generation method based on multispectral image reconstruction. It improves upon existing 3D reconstruction processes, enabling full utilization of multispectral information, robustness to areas with mixed vegetation, and enhanced accuracy of multispectral 3D reconstruction. The multispectral point cloud generation method based on multispectral image reconstruction described in this embodiment includes the following steps:
[0063] Step 1: Based on the multispectral camera and multispectral image acquisition parameters, perform radiometric correction and band alignment on the multispectral images, crop overlapping areas of the multispectral images, and superimpose multiple single-band images to synthesize a band-aligned multispectral image; specifically including the following steps:
[0064] (a) Radiation correction:
[0065] Based on the imaging principle of multispectral sensors, the mathematical model for radiometric correction of instantaneously acquired multispectral images is as follows:
[0066] L=g·V(x,y)·R(y)
[0067] Where L is the radiometric correction model for the multispectral image, g is the sensor gain coefficient, V(x,y) is the removal of edge light reduction effect caused by sensor lens, and R(y) is the row gradient correction factor.
[0068] The edge dimming effect V(x,y) caused by removing the sensor lens, i.e., the phenomenon where the sensor edge is darker than the perspective center, can be specifically described by the following formula:
[0069]
[0070] k = 1 + k0r + k1r 2 +k2r 3 +k3r 4 +k4r 5 +k5r 6
[0071]
[0072] Where k is the vignette correction factor, k i1 The coefficients of the correction polynomial are obtained from the sensor, i1 = 0, 1, ... 5, (c x ,c y () is the sensor vignetting correction center.
[0073] The row gradient correction factor R(y) is used to correct the readout noise of the sensor. Since the sensor reads data row by row, the sensor readings gradually decrease as the number of rows y increases. The correction model is as follows:
[0074]
[0075] Where I(x,y) is the original intensity value of pixel (x,y) in the image, I BL t represents the black level value of the image. e denoted as the exposure time of the multispectral image, and a1 and a2 as radiometric correction coefficients.
[0076] (II) Band Alignment: The purpose is to spatially register different bands of multispectral images acquired by a multispectral sensor. This mainly includes the following steps:
[0077] S1.2.1 Perform geometric distortion correction on band-by-band images:
[0078] x'=x(1+j1r 2 +j2r 4 +j3r 6 )
[0079] y'=y(1+j1r 2 +j2r 4 +j3r 6 )
[0080] Where (x', y') represents the corrected pixel position, j β β represents the lens distortion correction factor, where β = 1, 2, 3;
[0081] S1.2.2 Perform geometric registration transformation on images of each band using sensor lens structure parameters:
[0082] Since multispectral cameras use discrete multi-lens systems to acquire images across different spectral bands, there are certain differences in the center position and imaging angle between each lens. Using the sensor's factory parameters, the positional differences between the lenses and their Euler angles can be obtained. The Euler angles can then be converted into a rotation matrix using the following formula:
[0083]
[0084]
[0085]
[0086]
[0087] in, These are the three Euler angles between the shots.
[0088] By calculating the rotation matrix between lenses, geometric transformations can be performed on images of each band, enabling the registration and alignment of images acquired by multiple lenses.
[0089] (iii) The overlapping areas of the multispectral images are cropped, and multiple single-band images are superimposed to synthesize a multispectral image with band alignment.
[0090] Step 2: Extract features from the multispectral image using a multispectral fusion feature extraction algorithm.
[0091] Since multispectral images contain spectral information from multiple bands, using only one intensity value for feature extraction is insufficient to fully utilize this information. Therefore, this step combines image enhancement with the SIFT feature operator. The image enhancement employs the contrast-limited adaptive histogram equalization (CLAHE) algorithm, yielding the following band-by-band features:
[0092]
[0093]
[0094] Among them, I k For the k-th band of the multispectral image obtained in step one, CLAHE(·) represents the adaptive histogram equalization CLAHE algorithm processing. F represents the result of image enhancement. k The features extracted for the k-th band.
[0095] After all band features have been extracted, the following method is used to fuse the multispectral features:
[0096] For any multispectral image, iterate through the position of each pixel in the image. If there is only one multispectral feature at the pixel position, retain that feature as the fused multispectral feature. If there are multiple multispectral features at the pixel position, filter these features and retain the feature with the largest feature amplitude as the multispectral fusion feature.
[0097] The criterion for feature selection is based on the magnitude of the feature amplitude. During the calculation process, the SIFT operator retains the maximum gradient value in the principal direction of the feature point as the amplitude of that feature point, i.e., the SIFT feature amplitude. If there are multiple multispectral SIFT features in a pixel, the SIFT feature with the largest feature amplitude is retained as the multispectral fusion feature of that pixel.
[0098] Step 3: Calculate the NDVI value of the multispectral image and generate an NDVI mask. Use the NDVI mask to match and geometrically verify the multispectral features.
[0099] Using the multispectral image obtained in step 1, calculate the NDVI value of the multispectral image according to the following formula, and generate an NDVI mask;
[0100]
[0101]
[0102] Among them, I NDVI The NDVI value representing a multispectral image, I NIR For the near-infrared band of multispectral images, I R The red band value of the multispectral image; Mask V For NDVI masking, the value is 1 for vegetated areas and 0 for non-vegetated areas.
[0103] Based on the location of the NDVI mask and the fused multispectral features, the fused multispectral features are divided into F regions of vegetation areas. V Non-vegetated area F NV For these two features, the nearest neighbor algorithm is used for matching. In the feature matching process of multispectral images with overlapping regions (fusion of multispectral features), the vegetation region feature F in the reference image is used. V Vegetation region features in other multispectral images only F V Perform matching, but do not perform matching with non-vegetated area features F. NV The matching calculation. Similarly, the non-vegetated area features F in the reference image. NV It also only features non-vegetated areas F in neighboring multispectral images. NV Nearest neighbor matching is performed. Geometric verification is then conducted using random sample consistency to obtain feature matching pairs between multispectral images.
[0104] Step 4: Based on the obtained image feature matching relationship, use the motion structure recovery method to calculate the relative pose information of the image, use the multi-view geometric matching technique to calculate the image depth value, project the multispectral values, and generate a multispectral point cloud;
[0105] Based on the feature matching pairs obtained in step 3, the matching relationship of multispectral images in the image set is established. The relative pose relationship between multispectral images is obtained according to the triangulation principle. The motion structure recovery method is used to perform three-dimensional reconstruction of the multispectral images, obtaining the three-dimensional relationship of the reconstructed feature points and the image pose relationship optimized based on optimization criteria. The optimization criteria are as follows:
[0106]
[0107] Among them, M i X represents the projection matrix of the i-th camera. j Let x be the reconstructed coordinates of the j-th 3D reconstruction point. ij is the image coordinate of the j-th 3D reconstruction point in the i-th image; E(M,X) is the loss function optimized by the bundle adjustment method, minimizing it can minimize the reprojection error;
[0108] The multispectral image and the optimized pose relationship are used as input for multi-view stereo vision to estimate the depth map of the reconstructed image. The spatial position of the reconstructed point cloud is obtained by using the pose relationship of the image and the depth map of the multispectral image. The multispectral values are then assigned to the corresponding spatial points through projection, and finally, the multispectral point cloud MSPC is generated.
[0109]
[0110] Among them, (x i ,y i ,z i () represents the three-dimensional coordinates of the i-th point. p is the spectral value of the nth band at the i-th point. num It represents the total number of points in the multispectral point cloud.
[0111] The multispectral point cloud generation method designed in this invention was used to perform three-dimensional reconstruction of a multispectral image dataset acquired by a UAV. The reconstruction results are as follows: Figure 2 As shown in the figure. The drone used was a DJI M300, and the multispectral camera was a Micasense Rededge-MX. The data acquisition location was located in the Harbin Institute of Technology Science Park. A total of 1200 multispectral images were collected, with a reconstruction participation rate of 100%. The average number of extracted features was 24674, the number of reconstructed feature points was 160280, the reprojection error was 0.36 pixels, and the reconstruction error was 0.37m. The reconstruction accuracy is higher than that of existing methods. Specific Implementation Method Two:
[0113] This embodiment is a computer storage medium that stores at least one instruction, which is loaded and executed by a processor to implement the multispectral point cloud generation method based on multispectral image reconstruction.
[0114] It should be understood that the instructions include computer program products, software, or computerized methods corresponding to any method described in this invention; the instructions can be used to program computer systems or other electronic devices. Computer storage media may include readable media on which instructions are stored, and may include, but are not limited to, magnetic storage media, optical storage media; magneto-optical storage media include read-only memory (ROM), random access memory (RAM), erasable programmable memory (e.g., EPROM and EEPROM), and flash memory layers, or other types of media suitable for storing electronic instructions. Specific implementation method three:
[0116] This embodiment is a multispectral point cloud generation method and device based on multispectral image reconstruction. The device includes a processor and a memory. It should be understood that it includes any device including a processor and a memory described in this invention. The device may also include other units and modules that perform display, interaction, processing, control and other functions through signals or instructions.
[0117] The memory stores at least one instruction, which is loaded and executed by the processor to implement the multispectral point cloud generation method based on multispectral image reconstruction.
[0118] The above examples of the present invention are merely illustrative of the computational model and process of the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is impossible to exhaustively list all possible implementations here. Any obvious variations or modifications derived from the technical solutions of the present invention are still within the scope of protection of the present invention.
Claims
1. A method for generating multispectral point clouds based on multispectral image reconstruction, characterized in that, Includes the following steps: Step 1: Based on the multispectral camera and multispectral image acquisition parameters, perform radiometric correction and band alignment on the multispectral images; crop the overlapping areas of the multispectral images, and superimpose multiple single-band images to synthesize a band-aligned multispectral image. Step 2: Obtain band-by-band features by combining image enhancement with the SIFT feature operator. After all band features have been extracted, fuse the multispectral features using the following method: For any multispectral image, iterate through the position of each pixel in the image. If there is only one multispectral feature at the pixel position, retain that feature as the fused multispectral feature. If there are multiple multispectral features at the pixel position, filter these features and retain the feature with the largest feature amplitude as the multispectral fusion feature. Step 3: Based on the multispectral image obtained in Step 1, calculate the NDVI value of the multispectral image and generate an NDVI mask; use the NDVI mask to match and geometrically verify the multispectral features fused in Step 2. Step 4: Based on the feature matching pairs obtained in Step 3, establish the matching relationship of multispectral images in the image set, calculate the relative pose information of the images using the motion structure recovery method, calculate the image depth value using multi-view geometric matching technology, project the multispectral values, and generate a multispectral point cloud.
2. The method for generating multispectral point clouds based on multispectral image reconstruction according to claim 1, characterized in that, The radiation correction process in step 1 includes the following steps: Based on the imaging principle of multispectral sensors, the mathematical model for radiometric correction of instantaneously acquired multispectral images is as follows: L=g·V(x,y)·R(y) Where L is the radiometric correction model for the multispectral image, g is the sensor gain coefficient, V(x,y) is the removal of edge light reduction effect caused by sensor lens, and R(y) is the row gradient correction factor. The edge light reduction effect V(x,y) caused by removing the sensor lens is as follows: k=1+k0r+k1r 2 +k2r 3 +k3r 4 +k4r 5 +k5r 6 Where k is the vignette correction factor, k α The coefficients of the correction polynomial are obtained from the sensor, α = 0, 1, ... 5, (c x ,c y (This is the sensor vignette correction center;) The row gradient correction factor R(y) is as follows: Where I(x,y) is the original intensity value of pixel (x,y) in the image, I BL t represents the black level value of the image. e denoted as the exposure time of the multispectral image, and a1 and a2 as radiometric correction coefficients.
3. The method for generating multispectral point clouds based on multispectral image reconstruction according to claim 2, characterized in that, The band alignment process in step 1 includes the following steps: S1.2.1 Perform geometric distortion correction on band-by-band images: x'=x(1+j1r 2 +j2r 4 +j3r 6 ) y'=y(1+j1r 2 +j2r 4 +j3r 6 ) Where (x', y') represents the corrected pixel position, j β β represents the lens distortion correction factor, where β = 1, 2, 3; S1.2.2 Perform geometric registration transformation on images of each band using sensor lens structure parameters: Based on the positional differences between the lenses and the Euler angles, the Euler angles are converted into a rotation matrix using the following formula: in, The three Euler angles between the shots; By calculating the rotation matrix between lenses, geometric transformations can be performed on images of each band, enabling the registration and alignment of images acquired by multiple lenses.
4. A method for generating multispectral point clouds based on multispectral image reconstruction according to any one of claims 1 to 3, characterized in that, The band-by-band features obtained by combining image enhancement with the SIFT feature operator are as follows: Among them, I k For the k-th band of the multispectral image obtained in step one, CLAHE(·) represents the adaptive histogram equalization CLAHE algorithm processing. F represents the result of image enhancement. k The features extracted for the k-th band.
5. The method for generating multispectral point clouds based on multispectral image reconstruction according to claim 4, characterized in that, The process of calculating the NDVI value of the multispectral image based on the multispectral image obtained in step 1 and generating an NDVI mask includes the following steps: Based on the multispectral image obtained in step 1, the NDVI value of the multispectral image is calculated according to the following formula, and an NDVI mask is generated: Among them, I NDVI The NDVI value representing a multispectral image, I NIR For the near-infrared band of multispectral images, I R The red band value of the multispectral image; Mask V For NDVI masking, the value is 1 for vegetated areas and 0 for non-vegetated areas.
6. The method for generating multispectral point clouds based on multispectral image reconstruction according to claim 5, characterized in that, The process of matching and geometrically verifying the multispectral features fused in step 2 using an NDVI mask includes the following steps: Based on the location of the NDVI mask and the fused multispectral features, the fused multispectral features are divided into F regions of vegetation areas. V Non-vegetated area F NV For these two features, the nearest neighbor algorithm is used for matching. During the feature matching process, the vegetation region feature F in the reference image is used. V Vegetation region features in other multispectral images only F V Matching is performed using the non-vegetated region features F in the reference image. NV Features of non-vegetated areas in other multispectral images only F NV Nearest neighbor matching is performed; and geometric verification is performed using random sample consistency to obtain feature matching pairs between multispectral images.
7. The method for generating multispectral point clouds based on multispectral image reconstruction according to claim 6, characterized in that, The specific process of step 4 includes the following steps: Based on the feature matching pairs obtained in step 3, the matching relationship of multispectral images in the image set is established. The relative pose relationship between multispectral images is obtained according to the triangulation principle. The motion structure recovery method is used to perform three-dimensional reconstruction of the multispectral images, obtaining the three-dimensional relationship of the reconstructed feature points and the optimized image pose relationship based on optimization criteria. The optimization criteria are as follows: Among them, M i X represents the projection matrix of the i-th camera. j Let x be the reconstructed coordinates of the j-th 3D reconstruction point. ij is the image coordinate of the j-th 3D reconstruction point in the i-th image; E(M,X) is the loss function optimized by the bundle adjustment method, minimizing it can minimize the reprojection error; The multispectral image and the optimized pose relationship are used as input for multi-view stereo vision to estimate the depth map of the reconstructed image. The spatial position of the reconstructed point cloud is obtained by using the pose relationship of the image and the depth map of the multispectral image. The multispectral values are then assigned to the corresponding spatial points through projection, and finally, the multispectral point cloud MSPC is generated.
8. The method for generating multispectral point clouds based on multispectral image reconstruction according to claim 7, characterized in that, The multispectral point cloud MSPC is described below: Among them, (x i ,y i ,z i () represents the three-dimensional coordinates of the i-th point. p is the spectral value of the nth band at the i-th point. num It represents the total number of points in the multispectral point cloud.
9. A computer storage medium, characterized in that, The storage medium stores at least one instruction, which is loaded and executed by a processor to implement a multispectral point cloud generation method based on multispectral image reconstruction as described in any one of claims 1 to 8.
10. A multispectral point cloud generation device based on multispectral image reconstruction, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction, which is loaded and executed by the processor to implement a multispectral point cloud generation method based on multispectral image reconstruction as described in any one of claims 1 to 8.