Multi-band point cloud fusion method and device

Abstract

The present disclosure relates to the field of image processing, and provides a multi-band point cloud fusion method and device. The method can first collect three-dimensional point clouds of multiple bands, and obtain two-dimensional depth maps and two-dimensional intensity maps convenient for operation and processing through dimension conversion. Any band is taken as a reference band to register the remaining bands, and the corresponding registered depth maps and registered intensity maps of each remaining band are obtained. Based on the three-dimensional point cloud of the reference band, the registered depth maps and registered intensity maps of each remaining band are inversely mapped back to three dimensions, and the three-dimensional registered point cloud of each remaining band and the three-dimensional point cloud of the reference band are obtained. Based on the three-dimensional point cloud of the reference band, the intensity of each remaining band is accurately fused into the three-dimensional point cloud of the reference band. In this way, the obtained fusion result can eliminate the influence of position deviation, and the fusion result has higher accuracy. The registration process can improve the robustness of the fusion result, and facilitate subsequent point cloud analysis based on the fusion result.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a multi-band point cloud fusion method and device. Background Technology

[0002] In existing technologies, laser sensors of different wavelengths are used to scan the same area, thus obtaining point clouds of multiple wavelengths.

[0003] Point clouds from multiple bands are typically fused directly. However, because the control positions of laser sensors in different bands are set differently, the distribution of controls after fusion will have a large deviation, resulting in low accuracy. Summary of the Invention

[0004] In view of this, the purpose of this disclosure is to propose a multi-band point cloud fusion method and device to solve or partially solve the above-mentioned technical problems.

[0005] To achieve the above objectives, this disclosure provides a multi-band point cloud fusion method, including:

[0006] Acquire 3D point clouds across multiple wavelengths; For each band's 3D point cloud, the 3D point cloud of that band is dimensionally transformed to obtain the corresponding 2D depth map and 2D intensity map; Select any one of the multiple bands as a reference band, and register the two-dimensional depth map and two-dimensional intensity map of each remaining band with the reference band as a reference to obtain the registered depth map and registered intensity map corresponding to each remaining band. Each remaining band belongs to other bands in the multiple bands besides the reference band. The three-dimensional point cloud of the reference band is obtained. Based on the three-dimensional point cloud of the reference band, the registration depth map and registration intensity map corresponding to each remaining band are reverse mapped back to three dimensions to obtain the three-dimensional registration point cloud of each remaining band. Using the three-dimensional point cloud of the reference band as the master point cloud, and based on the master point cloud, the intensities corresponding to the three-dimensional registration point clouds of each remaining band are fused into the master point cloud to obtain the fusion result.

[0007] Based on the same inventive concept, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0008] As can be seen from the above, the multi-band point cloud fusion method and device provided in this disclosure can first acquire three-dimensional point clouds of multiple bands, so that the dimensional transformation of the three-dimensional point cloud of each band can be performed to obtain two-dimensional depth maps and two-dimensional intensity maps that are easy to process. Then, taking any band as a reference band, the two-dimensional depth map of the reference band can be used as a benchmark to register the two-dimensional depth maps and two-dimensional intensity maps of the other remaining bands, so as to obtain the registration depth maps and registration intensity maps corresponding to each remaining band. In order to facilitate three-dimensional point cloud fusion, it is also necessary to inversely map the registration depth maps and registration intensity maps corresponding to each remaining band back to three dimensions based on the three-dimensional point cloud of the reference band. In this way, the three-dimensional registration point clouds of each remaining band are matched with the three-dimensional point cloud of the reference band in terms of position. Finally, taking the three-dimensional point cloud of the reference band as the master point cloud, the three-dimensional registration point clouds of each remaining band are subjected to intensity fusion processing according to the accurate and consistent position, which can achieve the purpose of accurately fusing the intensity corresponding to each remaining band to the master point cloud. This ensures that the fusion result can eliminate the influence of positional offset, resulting in higher accuracy. Furthermore, the registration process described above improves the robustness of the fusion result, facilitating subsequent point cloud analysis based on it. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in this disclosure or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating the multi-band point cloud fusion method according to an embodiment of the present disclosure; Figure 2 This is a schematic diagram illustrating the conversion of a single-band point cloud (i.e., a single-band three-dimensional point cloud) into a depth map (two-dimensional depth map), an intensity map (two-dimensional intensity map), and corresponding spatial metadata according to an embodiment of this disclosure. Figure 3 This is an intention to demonstrate perspective transformation using a homography matrix (i.e., a transformation matrix) in an embodiment of this disclosure; Figure 4 This is a schematic diagram of the three-dimensional restoration (i.e., inverse mapping) process according to an embodiment of the present disclosure; Figure 5 This is a schematic diagram of information about a single point in the fusion result of an embodiment of this disclosure; Figure 6 This is a schematic diagram of the fusion result of an embodiment of this disclosure; Figure 7This is a structural block diagram of the multi-band point cloud fusion device according to an embodiment of the present disclosure; Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0012] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0013] In related technologies, with the rapid development of intelligent technologies, high-throughput and high-precision acquisition of three-dimensional phenotypes (especially plant phenotypes) has become an important research method. Linearly scanned laser sensors, with their high-precision and high-density three-dimensional point cloud acquisition capabilities, have shown broad application prospects in phenotypic research. To obtain comprehensive phenotypic information under different spectral bands, researchers often need to simultaneously deploy multiple linearly scanned laser sensors of different bands to scan the same area, thereby acquiring three-dimensional point cloud data corresponding to each band for subsequent multispectral phenotypic analysis.

[0014] However, due to the spacing between the physical installation locations of sensors of different wavelengths, even if all sensors scan the same area simultaneously, spatial positional deviations are still inevitable between the acquired point clouds of different wavelengths. In addition, different wavelengths of laser light have different penetration characteristics into objects (e.g., plants), and the object structures reflected by the point clouds of different wavelengths vary, resulting in a point distribution in three-dimensional space that is not completely consistent and shows significant differences.

[0015] Traditional 3D point cloud registration methods (such as the ICP (Iterative Closest Point) algorithm and its variants) have significant shortcomings in handling the above problems: On the one hand, the canopy without maps is chaotic (e.g., plant canopy growth is disordered), and the point cloud structure features are irregular, making it extremely difficult to find stable and reliable corresponding features for 3D registration, resulting in low registration robustness; on the other hand, the number of point clouds in a single band can reach millions, making the computational complexity of 3D point cloud registration extremely high and the amount of computation huge, making it difficult to meet the efficiency requirements of high-throughput phenotyping studies; more importantly, when the spatial distribution of point clouds in different bands differs greatly, 3D point cloud registration methods cannot overcome the problem of low point set overlap, and registration often fails or produces serious errors.

[0016] The embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0017] The multi-band point cloud fusion method proposed in the embodiments of this disclosure, such as Figure 1 As shown, it includes: Step 101: Collect three-dimensional point clouds of multiple bands.

[0018] In practice, for the target area (e.g., an area with vegetation, especially a field area with crops), a multi-band laser sensor device positioned above the target area can be used to collect a multi-band three-dimensional point cloud of the target area. The multi-band includes at least two of the following: red band, green band, blue band, red-edge band, and near-infrared band.

[0019] Step 102: For the 3D point cloud of each band, perform dimensional transformation on the 3D point cloud of that band to obtain the corresponding 2D depth map and 2D intensity map (e.g., ...). Figure 2 (As shown).

[0020] In practice, to facilitate calculations, the 3D point clouds of each band are transformed into a 2D space through dimensionality transformation, resulting in 2D depth maps and 2D intensity maps corresponding to each band. This process converts large amounts of 3D point cloud data into smaller amounts of 2D data, which facilitates subsequent registration processing.

[0021] Step 103: Select any one of the multiple bands as a reference band. Using the two-dimensional depth map of the reference band as a reference, register the two-dimensional depth map and two-dimensional intensity map of each remaining band to obtain the registered depth map and registered intensity map corresponding to each remaining band. Each remaining band belongs to other bands in the multiple bands besides the reference band.

[0022] In practice, any one of multiple bands needs to be selected as the reference band; the specific band chosen can be determined based on actual needs. This allows us to determine the transformation matrix of the remaining bands' 2D depth maps relative to the reference band's 2D depth map, based on the reference band's 2D depth map. Each remaining band then receives a corresponding transformation matrix. Subsequently, the 2D depth map and 2D intensity map of each remaining band are transformed using their respective transformation matrices to complete the position registration of the remaining bands to the reference band. The resulting registered depth map and registered intensity map for each remaining band will then match the position of the reference band.

[0023] Step 104: Obtain the 3D point cloud of the reference band. Based on the 3D point cloud of the reference band, reverse map the registration depth map and registration intensity map corresponding to each remaining band back to 3D to obtain the 3D registration point cloud of each remaining band.

[0024] In practice, to facilitate the subsequent fusion process, since the registration depth map and registration intensity map corresponding to each residual band are two-dimensional, they need to be converted back to three-dimensional. Therefore, based on the spatial position of the three-dimensional point cloud of the reference band, the registration depth map and registration intensity map corresponding to each residual band are converted into the corresponding three-dimensional registration point cloud.

[0025] Step 105: Using the three-dimensional point cloud of the reference band as the master point cloud, and based on the master point cloud, fuse the intensities corresponding to the three-dimensional registration point clouds of each remaining band into the master point cloud to obtain the fusion result.

[0026] In practice, after obtaining the three-dimensional registration point clouds of each remaining band, their corresponding intensities can be fused into the main point cloud according to their three-dimensional positions, thus obtaining an accurate fusion result. Based on this fusion result, accurate analysis can be performed.

[0027] The above scheme allows for the initial acquisition of 3D point clouds across multiple bands. This enables dimensionality transformation of the 3D point cloud for each band, yielding easily processed 2D depth and intensity maps. Then, using any band as a reference, the 2D depth map of the reference band serves as a benchmark for registering the 2D depth and intensity maps of the remaining bands, resulting in registered depth and intensity maps for each remaining band. To facilitate 3D point cloud fusion, the registered depth and intensity maps for each remaining band are inversely mapped back into 3D based on the reference band's 3D point cloud. This ensures that the resulting 3D registered point clouds for each remaining band are positionally matched to the 3D point cloud of the reference band. Finally, using the reference band's 3D point cloud as the master point cloud, the intensity of the 3D registered point clouds for each remaining band is fused according to their accurate and consistent positions, achieving the goal of accurately fusing the intensities corresponding to each remaining band into the master point cloud. This ensures that the fusion result can eliminate the influence of positional offset, resulting in higher accuracy. Furthermore, the registration process described above improves the robustness of the fusion result, facilitating subsequent point cloud analysis based on it.

[0028] In some embodiments, step 102 is performed for the three-dimensional point cloud for each band: Step 1021: Perform dimensional transformation on the three-dimensional point coordinates in the three-dimensional point cloud of this band to obtain two-dimensional point coordinates.

[0029] In practice, to facilitate dimensional transformation, any band is selected from the 3D point clouds of multiple bands as a reference band (for example, the R640 band, corresponding to the red band). Then, using the 3D point cloud of the reference band as the reference point cloud, the 3D point clouds of each band are orthogonally projected onto the reference point cloud to determine the 2D point coordinates corresponding to each 3D point coordinate.

[0030] For example, for the three-dimensional coordinate points in the three-dimensional point cloud of each band Its corresponding two-dimensional coordinates Calculated by the following formula: in The image resolution for the corresponding band (units consistent with 3D point cloud coordinates). , These are the minimum coordinate values ​​of the 3D point cloud in the X and Y directions.

[0031] Step 1022: Obtain the vertical linear quantity mapping of the 3D point cloud for this band as a depth value, and write the corresponding depth value to the corresponding 2D point coordinates to obtain a 2D depth map (e.g., ...). Figure 2 (Depth map shown).

[0032] In practice, after determining the vertical linear quantity in the three-dimensional point cloud for each band and mapping it to a depth value, the depth value can be written into the two-dimensional point coordinates obtained after the corresponding three-dimensional point cloud dimension conversion.

[0033] For example, for the i-th point, its corresponding 16-bit depth value The process of determining is as follows:

[0034] in, For the third point cloud The Z coordinates (i.e., vertical linear quantities) of each point; , These are the minimum and maximum values ​​of the 3D point cloud in the Z direction, respectively; , The minimum and maximum values ​​of the 16-bit pixel depth ( (Retained as invalid pixel identifier); For the first The depth value corresponding to each point; This indicates rounding to the nearest integer.

[0035] Step 1023: Obtain the intensity value corresponding to the coordinates of each three-dimensional point in the three-dimensional point cloud of the band, and write the corresponding intensity value to the corresponding two-dimensional point coordinates to obtain a two-dimensional intensity map.

[0036] In practice, the intensity value corresponding to the three-dimensional point coordinates is directly retrieved from the three-dimensional point cloud of that band, and this intensity value is written into the two-dimensional coordinates after the three-dimensional point coordinates are transformed, thus obtaining a two-dimensional intensity map (e.g., Figure 2 (Intensity diagram shown).

[0037] For example, the 8-bit intensity value corresponding to the 3D point coordinates is directly written into the corresponding 2D coordinates, and in order to save the spatial metadata of the 3D point cloud for each band (including... , , , , (e.g.) Figure 2 As shown in the figure, it is used for subsequent 3D point cloud reconstruction.

[0038] The above scheme can accurately transform the dimensions of 3D point clouds in various bands, completing the dimensionality reduction process. It can transform large-volume 3D point clouds into smaller-volume 2D ones, which facilitates subsequent registration processing.

[0039] In some embodiments, step 103 includes: Step 1031: For the two-dimensional depth maps of multiple bands, select any band as a reference band, use the two-dimensional depth map of the reference band as a reference depth map, and determine the transformation matrix of the two-dimensional depth maps of each remaining band relative to the reference depth map based on the reference depth map.

[0040] In some embodiments, step 1031 includes: Step 10311: Use the two-dimensional depth map of the reference band as the reference depth map, and use the two-dimensional depth map of each remaining band as the depth map to be registered.

[0041] Step 10312: Convert the reference depth map and each of the depth maps to be registered into a reference grayscale map with a predetermined number of bits and each of the grayscale maps to be registered.

[0042] In practice, the reference depth map and each depth map to be registered are used as target depth maps, and the bit depth is reduced according to a predetermined number of bits to obtain the corresponding grayscale image.

[0043] For example, a 16-bit target depth map can be normalized to obtain an 8-bit grayscale image to meet the input requirements of the feature description algorithm. The formula is as follows:

[0044] in, This represents the depth value of a pixel in a 16-bit target depth map. For all valid pixels in the target depth map ( The minimum depth value; The maximum depth value of all valid pixels in the target depth map; This is the 8-bit grayscale value after normalization; This indicates rounding to the nearest integer.

[0045] After each pixel in the target depth map is bit-converted to obtain its corresponding grayscale value, a grayscale image (including a reference grayscale image and each grayscale image to be registered) is obtained.

[0046] Step 10313, perform the following for each grayscale image to be registered: Step A1: Use a feature description algorithm (e.g., SIFT algorithm, Scale Invariant Feature Transform) to determine reference key points and reference descriptors corresponding to the reference key points, and combine all reference descriptors to form a reference descriptor set.

[0047] In practice, the SIFT algorithm is applied to the reference grayscale image to detect salient SIFT feature points (reference keypoints). For each SIFT feature point, a reference descriptor representing the local appearance surrounding that feature point is calculated. That is: Reference grayscale image → Reference key point set Reference descriptor subset .

[0048] Step A2: Use a feature description algorithm to determine the key points to be registered and the descriptors to be registered corresponding to the key points to be registered, and combine all the descriptors to be registered to form a set of descriptors to be registered.

[0049] In practice, the SIFT algorithm is executed on each grayscale image to be registered, detecting salient SIFT feature points (registration key points) in the image, and calculating the registration descriptor for the corresponding local features of each SIFT feature point, representing the local appearance around that SIFT feature point. That is: Grayscale image to be registered → Set of key points to be registered Descriptor subset to be registered .

[0050] Step A3: Using the nearest neighbor algorithm, select matching candidate pairs from the reference descriptor set and the descriptor set to be registered, and combine all matching candidate pairs to form a candidate pair set, wherein each matching candidate pair is a matching pair formed by a reference descriptor and a descriptor to be registered.

[0051] In some embodiments, in step A3, for each reference descriptor in the set of reference descriptors (e.g., For each reference descriptor, perform the filtering process: Step A31: In the set of descriptors to be registered, find the descriptor to be registered that is adjacent to the reference descriptor, and determine the nearest neighbor descriptor and the second nearest neighbor descriptor based on the distance between the adjacent descriptor to be registered and the reference descriptor.

[0052] Step A32: Determine the first distance between the nearest neighbor descriptor and the reference descriptor, and the second distance between the next nearest neighbor descriptor and the reference descriptor.

[0053] Step A33: In response to the ratio of the first distance to the second distance being less than the ratio threshold, the reference descriptor and the nearest neighbor descriptor are selected as a matching candidate pair.

[0054] In practice, in the subset of descriptors to be registered Find nearest neighbor descriptors ;exist Find the second nearest neighbor descriptor .

[0055] If satisfied (Lowe's Ratio Test) If τ is the ratio threshold, then the reference descriptor and the nearest neighbor descriptor are retained as reliable matching candidate pairs.

[0056] Step A34: After confirming that all reference descriptors in the reference descriptor set have been filtered, combine all the obtained matching candidate pairs to form a candidate pair set.

[0057] In practice, the corresponding candidate pair set is: , where, for the reference descriptor subset Find the corresponding nearest neighbor descriptor , where n is the total number of reference descriptors in the reference descriptor subset.

[0058] Using the above scheme, for each remaining band grayscale image to be registered, the nearest neighbor descriptor corresponding to the reference descriptor set determined based on the reference grayscale image can be accurately found. In this way, each remaining band grayscale image to be registered corresponds to a set of accurate candidate pairs, which facilitates further screening based on the candidate pair set to determine the corresponding transformation matrix.

[0059] Step A4: For the candidate pair set, the homography matrix is ​​determined using the random sample consensus algorithm, and the homography matrix is ​​used as the transformation matrix for the remaining band.

[0060] In some embodiments, step A4 is performed iteratively for the candidate pair set: Step A41: Extract a predetermined number of matching candidate pairs from the candidate pair set to obtain the extracted matching pair combination, and determine the candidate homography matrix based on the extracted matching pair combination.

[0061] In practice, in order to more accurately filter the candidate pair set, a predetermined number (e.g., 4) of matching candidate pairs will be extracted (e.g., randomly selected). Based on this predetermined number of matching candidate pairs, the candidate homography matrix H corresponding to the reference descriptor to the nearest neighbor descriptor will be determined.

[0062] Step A42: The pixels of the corresponding matching candidate pairs in the depth map to be registered are transformed according to the homography matrix, and the reprojection error between the coordinates of the transformed pixels and the coordinates of the corresponding pixels in the reference depth map is determined.

[0063] In practice, the candidate homography matrix H is applied to the corresponding matching candidate pixels in all depth maps to be registered, and the reprojection error between the transformed pixel coordinates and the corresponding pixel coordinates in the reference depth map is calculated.

[0064] Step A43: In response to the reprojection error being less than or equal to a preset threshold, the pixels of the corresponding matching candidate pair in the depth map to be registered are taken as interior points.

[0065] Step A44: In response to the reprojection error being greater than a preset threshold, the pixels of the corresponding matching candidate pair in the depth map to be registered are taken as outliers and the outliers are discarded.

[0066] In practice, pixel pairs in the depth map to be registered with a reprojection error less than a preset threshold are determined as inliers, which are reliable matching point pairs with consistent geometric relationships and conforming to transformation rules under the current candidate homography matrix H; otherwise, they are determined as outliers and discarded.

[0067] Step A45: Determine all interior points corresponding to the candidate homography matrix, and combine them to form an interior point set.

[0068] Step A46: In response to the number of iterations reaching a predetermined number (e.g., N times), the sets of inliers corresponding to each candidate homography matrix are compared, and the candidate homography matrix corresponding to the set of inliers with the largest number of inliers is selected as the transformation matrix.

[0069] The above method allows for further screening of the candidate pair set, thereby obtaining a transformation matrix capable of accurate position registration.

[0070] Step 1032: Perform conversion and registration processing on the two-dimensional depth map and two-dimensional intensity map of each remaining band according to the conversion matrix to obtain the registration depth map and registration intensity map corresponding to each remaining band.

[0071] In some embodiments, step 1032 performs the following for each remaining band: Step 10321: Based on the pixel coordinates of each pixel in the two-dimensional depth map corresponding to the remaining band, a depth map matrix is ​​formed. The transformation matrix is ​​multiplied by the depth map matrix to obtain the registration depth map corresponding to the remaining band.

[0072] For example, for each pixel coordinate in the 16-bit two-dimensional depth map corresponding to the remaining bands, a 16-bit depth map matrix is ​​formed, corresponding to the coordinates of each pixel point. A perspective transformation is performed using the transformation matrix H, mapping it to the coordinate system of the reference depth map corresponding to the reference band. The specific formula is as follows: ;in, For real numbers, The corresponding position in the coordinate system of the registered depth map after transformation (e.g.) Figure 3 (As shown).

[0073] Step 10322: Based on the pixel coordinates of each pixel in the two-dimensional intensity map corresponding to the remaining band, form an intensity map matrix, and multiply the transformation matrix with the intensity map matrix to obtain the registration intensity map corresponding to the remaining band.

[0074] For example, for each pixel coordinate of the 8-bit two-dimensional intensity map corresponding to the remaining bands, an 8-bit intensity map matrix is ​​formed, corresponding to the coordinates of each pixel point. A perspective transformation is performed using the transformation matrix H to map the image to the coordinate system of the reference intensity map corresponding to the reference band. The specific formula is as follows: ;in, This represents the corresponding position in the coordinate system of the registration intensity map after transformation.

[0075] Using the above method, the two-dimensional depth map and two-dimensional intensity map corresponding to the remaining bands can be transformed into a registered depth map and registered intensity map belonging to the same coordinate system as the reference band through a transformation matrix.

[0076] In some embodiments, in step 104, for each remaining band, the following is performed: Step 1041: Obtain the three-dimensional point cloud of the reference band, and determine the depth point cloud features and intensity point cloud features of the three-dimensional point cloud of the reference band.

[0077] In practical implementation, the corresponding deep point cloud features include: , To reference the minimum coordinate values ​​of the point cloud in the X and Y directions, , The minimum and maximum values ​​of the reference band point cloud in the Z direction.

[0078] Intensity point cloud features include: , The minimum and maximum intensity values ​​in the reference band point cloud.

[0079] Step 1042: According to the depth point cloud features, the registered depth map of the remaining band is inversely mapped back to the three-dimensional space corresponding to the depth point cloud features to obtain the mapped three-dimensional depth map.

[0080] In practical implementation, the corresponding first inverse mapping formula is:

[0081]

[0082] in, , To register the row and column coordinates of pixels in the depth map; Image resolution for registering depth maps; , The minimum coordinate values ​​of the reference band point cloud in the X and Y directions; To register the depth value of pixel i in the depth map; , The minimum and maximum values ​​for the depth value quantization range; , The minimum and maximum values ​​of the reference band point cloud in the Z direction; , , To recover the third dimension depth map The three-dimensional coordinates of each pixel.

[0083] Step 1043: According to the intensity point cloud features, the registered intensity map of the remaining band is inversely mapped back to the three-dimensional region corresponding to the intensity point cloud features to obtain the mapped three-dimensional intensity map.

[0084] In practical implementation, the corresponding second inverse mapping formula is:

[0085]

[0086] in, , To register the row and column coordinates of pixels in the intensity map; Image resolution for registering intensity maps; , The minimum coordinate values ​​of the reference band point cloud in the X and Y directions; To register the intensity value of pixel i in the intensity map; =255 is the maximum value in the intensity value quantization range; , The minimum and maximum values ​​of the point cloud intensity for the reference band; , , To recover the third 3D intensity map The three-dimensional coordinates of each pixel.

[0087] Step 1044: The mapped 3D depth map and the mapped 3D intensity map are fused according to 3D coordinates to obtain the 3D registration point cloud corresponding to the remaining band (e.g., ...). Figure 4 (As shown).

[0088] The above method requires converting the registration depth map and registration intensity map corresponding to each remaining band from two dimensions back to three dimensions to facilitate the subsequent fusion process.

[0089] In some embodiments, in step 105, the following is performed for each principal point in the principal point cloud of the three-dimensional point cloud of the reference band: Step 1051: Determine the two-dimensional plane corresponding to the principal point.

[0090] Step 1052: Find at least one nearest neighbor point in the two-dimensional plane whose distance is less than the predetermined search radius in each three-dimensional registration point cloud.

[0091] Step 1053: Determine the mean of the intensity values ​​corresponding to at least one nearest neighbor point, and use the mean of the intensity values ​​as the fusion intensity value corresponding to the master point.

[0092] Step 1054: After obtaining the fusion intensity values ​​corresponding to all principal points in the main point cloud, mark the fusion intensity values ​​on the corresponding principal points in the main point cloud to obtain the fusion result.

[0093] In practice, the 3D point cloud of the reference band is used as the main point cloud, and the 3D registered point clouds of each band are used as auxiliary point clouds. The intensity information of each band is sequentially fused into the main point cloud. The fusion process only involves performing a nearest neighbor search (ignoring the Z direction) on the auxiliary point cloud of any band within the 2D plane (XY plane) corresponding to the principal point in the main point cloud. For each principal point in the main point cloud... In the auxiliary point cloud, find points in the XY plane where the distance is less than the search radius. of For each nearest neighbor point, the average intensity of these nearest neighbors is taken as the fusion intensity value of the principal point in the corresponding band. For the auxiliary point cloud corresponding to band B, the formula for calculating the fusion intensity value is as follows: in, In the main point cloud (reference band point cloud) the first The coordinates of the principal points in the XY plane; To assist in the point cloud The coordinates of a point in the XY plane; The search radius; To assist in the point cloud and the first The distance between the points in the XY plane is less than The set of nearest neighbors; To assist in the point cloud Nearest neighbor points in the band The strength value below; The first point cloud The main point is in the band. The fusion intensity value is calculated by taking the average of the intensity values ​​of all nearest neighbor points.

[0094] If a principal point in the principal point cloud has no effective nearest neighbor within the search radius, the intensity value of the corresponding band of that principal point is recorded as 0.

[0095] For each band, the above scheme is executed once. After the fusion process is completed for all auxiliary point clouds of the remaining bands, each principal point in the principal point cloud simultaneously carries Band1 (intensity value 1 of band 1), Band2 (intensity value 2 of band 2), ..., Band n (Intensity value n of band n) Data of the fused intensity values ​​of multiple bands (e.g., Figure 5 As shown), a complete multi-band 3D point cloud fusion result is formed (e.g. Figure 6 (As shown).

[0096] The above scheme can accurately complete the fusion process of three-dimensional point clouds in various bands, ensuring higher accuracy of the final fusion result, improving the robustness of the fusion result, and facilitating subsequent point cloud analysis based on the fusion result.

[0097] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0098] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the claims of this disclosure. In some cases, the actions or steps described in the claims of this disclosure may be performed in a different order than that shown in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0099] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides a multi-band point cloud fusion device.

[0100] refer to Figure 7 The device includes: The acquisition module 201 is configured to acquire three-dimensional point clouds of multiple bands; The dimension transformation module 202 is configured to perform dimension transformation on the three-dimensional point cloud of each band to obtain the corresponding two-dimensional depth map and two-dimensional intensity map. The registration module 203 is configured to select any one of the multiple bands as a reference band, and register the two-dimensional depth map and two-dimensional intensity map of each remaining band with the two-dimensional depth map of the reference band as a reference, so as to obtain the registration depth map and registration intensity map corresponding to each remaining band, wherein each remaining band belongs to other bands in the multiple bands other than the reference band; The inverse mapping module 204 is configured to acquire the three-dimensional point cloud of the reference band, and based on the three-dimensional point cloud of the reference band, inversely map the registration depth map and registration intensity map corresponding to each remaining band back to three dimensions to obtain the three-dimensional registration point cloud of each remaining band. The fusion module 205 is configured to use the three-dimensional point cloud of the reference band as the main point cloud, and use the main point cloud as a reference to fuse the intensities corresponding to the three-dimensional registration point clouds of each remaining band into the main point cloud to obtain the fusion result.

[0101] In some embodiments, the dimension transformation module 202 is specifically configured as follows: Perform the following for each band of the 3D point cloud: The dimensionality of the 3D point coordinates in the 3D point cloud of this band is transformed to obtain the 2D point coordinates; The vertical linear quantities in the 3D point cloud of this band are mapped to depth values, and the corresponding depth values ​​are written to the corresponding 2D point coordinates to obtain a 2D depth map. Obtain the intensity value corresponding to the coordinates of each three-dimensional point in the three-dimensional point cloud of this band, and write the corresponding intensity value to the corresponding two-dimensional point coordinates to obtain a two-dimensional intensity map.

[0102] In some embodiments, the registration module 203 is specifically configured as follows: For two-dimensional depth maps of multiple bands, any band is selected as a reference band, and the two-dimensional depth map of the reference band is used as a reference depth map. Based on the reference depth map, the transformation matrix of the two-dimensional depth map of each remaining band relative to the reference depth map is determined. The two-dimensional depth map and two-dimensional intensity map of each remaining band are transformed and registered according to the transformation matrix to obtain the registered depth map and registered intensity map corresponding to each remaining band.

[0103] In some embodiments, the registration module 203 is further configured to: The two-dimensional depth map of the reference band is used as the reference depth map, and the two-dimensional depth map of each remaining band is used as the depth map to be registered. The reference depth map and each of the depth maps to be registered are converted into a reference grayscale map with a predetermined number of bits and each of the grayscale maps to be registered. Perform the following for each grayscale image to be registered: The reference grayscale image is used to determine reference key points and reference descriptors corresponding to the reference key points, and all reference descriptors are combined to form a reference descriptor set. The grayscale image to be registered is used to determine the key points to be registered and the descriptors to be registered corresponding to the key points to be registered. All the descriptors to be registered are combined to form a set of descriptors to be registered. The nearest neighbor algorithm is used to filter matching candidate pairs from the reference descriptor set and the descriptor set to be registered, and all matching candidate pairs are combined to form a candidate pair set, wherein each matching candidate pair is a matching pair formed by a reference descriptor and a descriptor to be registered; For the candidate pair set, a homography matrix is ​​determined using a random sample consensus algorithm, and the homography matrix is ​​used as the transformation matrix.

[0104] In some embodiments, the registration module 203 is further configured to: For each reference descriptor in the aforementioned set of reference descriptors, a filtering process is performed: In the set of descriptors to be registered, find the descriptor to be registered that is adjacent to the reference descriptor, and determine the nearest neighbor descriptor and the second nearest neighbor descriptor based on the distance between the adjacent descriptor to be registered and the reference descriptor; Determine a first distance between the nearest neighbor descriptor and the reference descriptor, and a second distance between the second nearest neighbor descriptor and the reference descriptor; If the ratio of the first distance to the second distance is less than the ratio threshold, the reference descriptor and the nearest neighbor descriptor are selected as a matching candidate pair. Once all reference descriptors in the reference descriptor set have been filtered, all the resulting matching candidate pairs are combined to form a candidate pair set.

[0105] In some embodiments, the registration module 203 is further configured to: For the set of candidate pairs, iteratively execute: A predetermined number of matching candidate pairs are extracted from the candidate pair set to obtain an extracted matching pair combination, and a candidate homography matrix is ​​determined based on the extracted matching pair combination; The corresponding matching candidate pixel points in the depth map to be registered are transformed according to the homography matrix, and the reprojection error between the transformed pixel coordinates and the corresponding pixel coordinates in the reference depth map is determined. When the reprojection error is less than or equal to a preset threshold, the pixels of the corresponding matching candidate pair in the depth map to be registered are taken as inliers. When the reprojection error is greater than a preset threshold, the pixels of the corresponding matching candidate pair in the depth map to be registered are taken as out points and the out points are discarded. Determine all interior points corresponding to the candidate homography matrix, and combine them to form an interior point set; In response to the predetermined number of iterations, the sets of interior points corresponding to each candidate homography matrix are compared, and the candidate homography matrix corresponding to the set of interior points with the largest number of interior points is selected as the transformation matrix.

[0106] In some embodiments, the registration module 203 is further configured to: For each remaining band, execute: A depth map matrix is ​​formed based on the pixel coordinates of the two-dimensional depth map corresponding to the remaining band. The transformation matrix is ​​multiplied by the depth map matrix to obtain the registration depth map corresponding to the remaining band. An intensity map matrix is ​​formed based on the pixel coordinates of the two-dimensional intensity map corresponding to the remaining band. The transformation matrix is ​​multiplied by the intensity map matrix to obtain the registration intensity map corresponding to the remaining band.

[0107] In some embodiments, the inverse mapping module 204 is specifically configured as follows: For each remaining band, execute: Acquire the 3D point cloud of the reference band, determine the depth point cloud features of the 3D point cloud of the reference band, and the intensity point cloud features of the 3D point cloud of the reference band. Based on the depth point cloud features, the registered depth map of the remaining band is inversely mapped back to the 3D corresponding to the depth point cloud features to obtain the mapped 3D depth map. Based on the intensity point cloud features, the registered intensity map of the remaining band is inversely mapped back to the three-dimensional region corresponding to the intensity point cloud features to obtain the mapped three-dimensional intensity map. The mapped 3D depth map and the mapped 3D intensity map are fused according to the 3D coordinates to obtain the 3D registration point cloud corresponding to the remaining band.

[0108] In some embodiments, the fusion module 205 is specifically configured as follows: Using the main point cloud in the three-dimensional point cloud of the reference band, perform the following for each principal point in the main point cloud: Determine the two-dimensional plane corresponding to the principal point; In each 3D registration point cloud, find at least one nearest neighbor point in the 2D plane whose distance is less than a predetermined search radius; Determine the mean of the intensity values ​​corresponding to at least one nearest neighbor point, and use the mean of the intensity values ​​as the fusion intensity value corresponding to the master point; After obtaining the fusion intensity values ​​corresponding to all principal points in the main point cloud, the fusion intensity values ​​are marked on the corresponding principal points in the main point cloud to obtain the fusion result.

[0109] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0110] The apparatus of the above embodiments is used to implement the corresponding method in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0111] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any of the above embodiments.

[0112] Figure 8 This embodiment illustrates a more specific hardware structure of an electronic device. The device may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0113] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0114] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0115] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0116] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0117] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0118] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0119] The electronic devices described above are used to implement the corresponding methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0120] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to perform the methods described in any of the above embodiments.

[0121] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0122] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to perform the methods described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0123] Based on the same concept, corresponding to any of the above embodiments, this disclosure also provides a computer program product, including computer program instructions, which, when run on a computer, cause the computer to perform the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0124] It is understood that before using the technical solutions of the various embodiments in this disclosure, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0125] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.

[0126] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0127] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0128] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this disclosure as described above, which are not provided in detail for the sake of brevity.

[0129] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this disclosure, the provided drawings may or may not show well-known power / ground connections to integrated circuit (IC) chips and other components. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this disclosure, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this disclosure will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that the embodiments of this disclosure can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0130] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0131] The embodiments disclosed herein are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the claims of this disclosure. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A multi-band point cloud fusion method, characterized in that, include: Acquire 3D point clouds across multiple wavelengths; For each band's 3D point cloud, the 3D point cloud of that band is dimensionally transformed to obtain the corresponding 2D depth map and 2D intensity map; Select any one of the multiple bands as a reference band. Using the two-dimensional depth map of the reference band as a reference, register the two-dimensional depth map and two-dimensional intensity map of each remaining band to obtain the registered depth map and registered intensity map corresponding to each remaining band. Each remaining band belongs to other bands in the multiple bands besides the reference band. The three-dimensional point cloud of the reference band is obtained. Based on the three-dimensional point cloud of the reference band, the registration depth map and registration intensity map corresponding to each remaining band are reverse mapped back to three dimensions to obtain the three-dimensional registration point cloud of each remaining band. Using the three-dimensional point cloud of the reference band as the master point cloud, and based on the master point cloud, the intensities corresponding to the three-dimensional registration point clouds of each remaining band are fused into the master point cloud to obtain the fusion result. For each band of the 3D point cloud, the 3D point cloud of that band is dimensionally transformed to obtain the corresponding 2D depth map and 2D intensity map, including: Perform the following for each band of the 3D point cloud: The dimensionality of the 3D point coordinates in the 3D point cloud of this band is transformed to obtain the 2D point coordinates; The vertical linear quantities in the 3D point cloud of this band are mapped to depth values, and the corresponding depth values ​​are written to the corresponding 2D point coordinates to obtain a 2D depth map. Obtain the intensity value corresponding to the coordinates of each three-dimensional point in the three-dimensional point cloud of this band, and write the corresponding intensity value to the corresponding two-dimensional point coordinates to obtain a two-dimensional intensity map.

2. The method according to claim 1, characterized in that, The step of selecting any one of multiple bands as a reference band, and using the two-dimensional depth map of the reference band as a benchmark, registering the two-dimensional depth map and two-dimensional intensity map of each remaining band to obtain the registered depth map and registered intensity map corresponding to each remaining band includes: For two-dimensional depth maps of multiple bands, any band is selected as a reference band, and the two-dimensional depth map of the reference band is used as a reference depth map. Based on the reference depth map, the transformation matrix of the two-dimensional depth map of each remaining band relative to the reference depth map is determined. The two-dimensional depth map and two-dimensional intensity map of each remaining band are transformed and registered according to the transformation matrix to obtain the registered depth map and registered intensity map corresponding to each remaining band.

3. The method according to claim 2, characterized in that, The step of using the two-dimensional depth map of the reference band as a reference depth map, and using the reference depth map as a reference, to determine the transformation matrix of the two-dimensional depth map of each remaining band relative to the reference depth map, includes: The two-dimensional depth map of the reference band is used as the reference depth map, and the two-dimensional depth map of each remaining band is used as the depth map to be registered. The reference depth map and each of the depth maps to be registered are converted into a reference grayscale map with a predetermined number of bits and each of the grayscale maps to be registered. Perform the following for each grayscale image to be registered: The reference grayscale image is used to determine reference key points and reference descriptors corresponding to the reference key points, and all reference descriptors are combined to form a reference descriptor set. The grayscale image to be registered is used to determine the key points to be registered and the descriptors to be registered corresponding to the key points to be registered. All the descriptors to be registered are combined to form a set of descriptors to be registered. The nearest neighbor algorithm is used to filter matching candidate pairs from the reference descriptor set and the descriptor set to be registered, and all matching candidate pairs are combined to form a candidate pair set, wherein each matching candidate pair is a matching pair formed by a reference descriptor and a descriptor to be registered; For the candidate pair set, a homography matrix is ​​determined using a random sample consensus algorithm, and the homography matrix is ​​used as the transformation matrix.

4. The method according to claim 3, characterized in that, The process of using the nearest neighbor algorithm to filter matching candidate pairs from the reference descriptor subset and the descriptor subset to be registered, and combining all matching candidate pairs to form a candidate pair set, includes: For each reference descriptor in the aforementioned set of reference descriptors, a filtering process is performed: In the set of descriptors to be registered, find the descriptor to be registered that is adjacent to the reference descriptor, and determine the nearest neighbor descriptor and the second nearest neighbor descriptor based on the distance between the adjacent descriptor to be registered and the reference descriptor; Determine a first distance between the nearest neighbor descriptor and the reference descriptor, and a second distance between the second nearest neighbor descriptor and the reference descriptor; If the ratio of the first distance to the second distance is less than the ratio threshold, the reference descriptor and the nearest neighbor descriptor are selected as a matching candidate pair. Once all reference descriptors in the reference descriptor set have been filtered, all the resulting matching candidate pairs are combined to form a candidate pair set.

5. The method according to claim 3, characterized in that, The step of determining the homography matrix using a random sample consensus algorithm for the candidate pair set, and using the homography matrix as the transformation matrix, includes: For the set of candidate pairs, iteratively execute: A predetermined number of matching candidate pairs are extracted from the candidate pair set to obtain an extracted matching pair combination, and a candidate homography matrix is ​​determined based on the extracted matching pair combination; The corresponding matching candidate pixel points in the depth map to be registered are transformed according to the homography matrix, and the reprojection error between the transformed pixel coordinates and the corresponding pixel coordinates in the reference depth map is determined. When the reprojection error is less than or equal to a preset threshold, the pixels of the corresponding matching candidate pair in the depth map to be registered are taken as inliers. When the reprojection error is greater than a preset threshold, the pixels of the corresponding matching candidate pair in the depth map to be registered are taken as out points and the out points are discarded. Determine all interior points corresponding to the candidate homography matrix, and combine them to form an interior point set; In response to the predetermined number of iterations, the sets of interior points corresponding to each candidate homography matrix are compared, and the candidate homography matrix corresponding to the set of interior points with the largest number of interior points is selected as the transformation matrix.

6. The method according to claim 2, characterized in that, The step of performing transformation and registration processing on the two-dimensional depth map and two-dimensional intensity map of each remaining band according to the transformation matrix to obtain the registered depth map and registered intensity map corresponding to each remaining band includes: For each remaining band, execute: A depth map matrix is ​​formed based on the pixel coordinates of the two-dimensional depth map corresponding to the remaining band. The transformation matrix is ​​multiplied by the depth map matrix to obtain the registration depth map corresponding to the remaining band. An intensity map matrix is ​​formed based on the pixel coordinates of the two-dimensional intensity map corresponding to the remaining band. The transformation matrix is ​​multiplied by the intensity map matrix to obtain the registration intensity map corresponding to the remaining band.

7. The method according to claim 1, characterized in that, The process of acquiring the 3D point cloud of the reference band involves, based on the 3D point cloud of the reference band, inversely mapping the registration depth map and registration intensity map corresponding to each remaining band back to 3D, to obtain the 3D registration point cloud of each remaining band, including: For each remaining band, execute: Acquire the 3D point cloud of the reference band, determine the depth point cloud features of the 3D point cloud of the reference band, and the intensity point cloud features of the 3D point cloud of the reference band. Based on the depth point cloud features, the registered depth map of the remaining band is inversely mapped back to the 3D corresponding to the depth point cloud features to obtain the mapped 3D depth map. Based on the intensity point cloud features, the registered intensity map of the remaining band is inversely mapped back to the three-dimensional region corresponding to the intensity point cloud features to obtain the mapped three-dimensional intensity map. The mapped 3D depth map and the mapped 3D intensity map are fused according to the 3D coordinates to obtain the 3D registration point cloud corresponding to the remaining band.

8. The method according to claim 1, characterized in that, The process involves using the three-dimensional point cloud of the reference band as the master point cloud, and using the master point cloud as a reference, fusing the intensities corresponding to the three-dimensional registration point clouds of each remaining band into the master point cloud to obtain a fusion result, including: Using the main point cloud in the three-dimensional point cloud of the reference band, perform the following for each principal point in the main point cloud: Determine the two-dimensional plane corresponding to the principal point; In each 3D registration point cloud, find at least one nearest neighbor point in the 2D plane whose distance is less than a predetermined search radius; Determine the mean of the intensity values ​​corresponding to at least one nearest neighbor point, and use the mean of the intensity values ​​as the fusion intensity value corresponding to the master point; After obtaining the fusion intensity values ​​corresponding to all principal points in the main point cloud, the fusion intensity values ​​are marked on the corresponding principal points in the main point cloud to obtain the fusion result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 8.

Images