Laser point cloud and multispectral image fusion method based on Hough transform and semantic feature points

Abstract

The present invention discloses a method for fusing laser point cloud and multispectral image by combining Hough transform and semantic feature points, including: inputting laser point cloud and multispectral image into Line‑CNN deep learning network to extract corresponding line features, obtaining point cloud feature straight line graph and multispectral feature straight line graph; mapping feature straight line graph into Hough space, obtaining rectangular centroid and attribute information of point cloud and rectangular centroid and attribute information of multispectral through peak point detection, as semantic feature points; performing rectangular matching on semantic feature points, calculating projection matrix according to rectangular matching result, assigning multispectral information to laser point cloud according to projection matrix, finally obtaining laser point cloud with multispectral information, and realizing accurate fusion of laser point cloud and multispectral image. The present invention significantly improves the robustness and robustness of bimodal data matching and fusion, and effectively solves the problem of cross-modal data matching and fusion in complex ground scene.

Description

Technical Field

[0001] The invention belongs to the field of remote sensing technology, and in particular relates to a laser point cloud and multispectral image fusion method combining Hough transform and semantic feature points. Background Art

[0002] With the acceleration of urbanization, the demand for accurate identification and classification of urban objects, environmental monitoring, and cultural relics recording and protection is increasing. As an important means of modern remote sensing technology, airborne laser point cloud and multispectral imagery provide rich data support for obtaining fine features of cities and environments, as well as cultural relics information. However, achieving efficient and robust fusion of these two data sources to extract more accurate features is still a key challenge that needs to be solved in the field of remote sensing.

[0003] Among the common cross-modal data fusion methods, feature-level fusion methods are the most widely used. For example, Wang Mei et al. used the SIFT feature point matching algorithm to register moving images, and then used the differential multiplication method to realize the registration and detection of the moving target contour; Wen Gang et al. proposed an image fusion method based on WAResNet and GS+ATPRK, using the GS+ATPRK algorithm to achieve high-fidelity image fusion, and using the WAResNet network to improve the fusion speed and effect; Fu Yukai et al. adopted an image fusion algorithm that combines non-local self-similarity and divergence, using non-local directional entropy and divergence as key features to guide the fusion of low-frequency and high-frequency parts respectively; Hou Zhaoyang et al. decomposed the brightness component Y through non-subspace sparse transform (NSST), fused the low-frequency subbands using low-level visual feature rules, and used the PAPCNN model combined with multi-scale morphological gradient processing for the high-frequency subbands, and finally obtained the fused image through inverse transformation; Li et al. proposed an advanced deep learning architecture IVFuseNet, which can intelligently integrate feature information from different images and dynamically adjust parameters to improve the quality of the final image. However, due to the large amount of feature data, its time efficiency is low, and it still cannot solve problems such as geometric correction in image matching. In view of the above problems, it is urgent to propose a new method for fusing laser point cloud and multispectral image to improve the robustness and accuracy of multimodal data fusion, thereby improving the accuracy of object recognition and classification. Summary of the invention

[0004] The technical problem to be solved by the present invention is to address the deficiencies of the above-mentioned prior art and provide a laser point cloud and multispectral image fusion method combining Hough transform and semantic feature points, so as to achieve high accuracy and robust cross-modal fusion of laser point cloud and multispectral image in complex terrain scenes through intelligent data fusion and geographic information processing.

[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0006] The laser point cloud and multispectral image fusion method combining Hough transform and semantic feature points includes:

[0007] Step 1: Input the laser point cloud image and multispectral image into the Line-CNN deep learning network to extract the line features of the corresponding images, and obtain the point cloud feature straight line graph and the multispectral feature straight line graph;

[0008] Step 2: Map the point cloud feature straight line graph and the multispectral feature straight line graph to the Hough space, and obtain the rectangular centroid and attribute information of the point cloud and the rectangular centroid and attribute information of the multispectral through peak point detection as the semantic feature points of the point cloud and the multispectral semantic feature points;

[0009] Step 3: Rectangular matching is performed on the semantic feature points of the point cloud and the multispectral semantic feature points. The projection matrix is ​​calculated based on the rectangular matching results. The multispectral information is assigned to the laser point cloud based on the projection matrix. Finally, a laser point cloud with multispectral information is obtained, realizing the precise fusion of the laser point cloud and the multispectral image.

[0010] To optimize the above technical solutions, the specific measures taken also include:

[0011] The above step 1 includes:

[0012] Step 101: rasterizing the laser point cloud image to convert it into a two-dimensional raster laser point cloud image;

[0013] Step 102: Line feature extraction of laser point cloud images and multispectral images is performed based on the four stages of feature extraction, node prediction, line segment sampling, and line segment correction of the Line-CNN neural network. The line feature results are drawn on a blank image of the same size as the original image to obtain a point cloud feature straight line graph and a multispectral feature straight line graph.

[0014] In the above step 2, the point cloud feature straight line graph and the multispectral feature straight line graph are respectively mapped to the Hough space, all peak points greater than the threshold in the Hough space are counted, several pairs of approximately parallel and equal straight line segments are found, and then all pairs of straight line segments are compared to match the four sides of the rectangle, remove pseudo rectangles, return the center of gravity of the rectangle and assign it various attribute information of the corresponding rectangle as the semantic feature point.

[0015] The above statistics of all peak points in the Hough space that are greater than the threshold value finds several pairs of approximately parallel and equal straight line segments, including:

[0016] Assume that a pair of peak points are HS i (ρ i ,θ i ), HS j (ρ j ,θ j), the corresponding peak point cumulative value is Peak HSi (ρ i ,θ i ),Peak HSj (ρ j ,θ j ), if the following equation is satisfied, it is matched as a pair of approximately parallel and equal straight line segments:

[0017]

[0018] Where V θ 、V Peak is the positive threshold;

[0019] The two peak points satisfying equation (1) are marked as

[0020] The parameters

[0021] The above compares all pairs of straight line segments and matches the four sides of the rectangle, including:

[0022] Compare all The following conditions will be met Match the four sides of the rectangle:

[0023]

[0024] in, is the angle threshold.

[0025] The above-mentioned pseudo-rectangle removal is as follows:

[0026] The coordinates of the four vertices of the rectangle are obtained based on the two sets of opposite sides of the rectangle, A1, A2, A3, and A4. The number of pixels near the four sides is counted and compared with the corresponding side lengths of the rectangle. If any side does not meet the following conditions, it is judged as a pseudo-rectangle and removed:

[0027] |P ij -ζ k |<V P ζ k (3)

[0028] Where P ij For side A i A j The number of nearby pixels, V P is the ratio threshold.

[0029] The above step 3 performs rectangular matching on the point cloud semantic feature points and the multispectral semantic feature points, including:

[0030] All semantic feature points are exhaustively arranged and combined into triangles, the similarity between triangles is calculated, and the two triangles that are most similar and second most similar to each triangle are found. The triangles that meet constraint 1 are established in correspondence and stored in the matching set. Finally, the triangle pairs in the matching set are grouped according to constraint 2. The group with more matching triangle pairs is more likely to contain true matching pairs. The group with the largest number of matching pairs is taken as the result of rectangle matching.

[0031] The calculation formula for the similarity between the above triangles is:

[0032]

[0033] in, and are the triangle sets composed of point cloud semantic feature points and multispectral semantic feature points, respectively, |Δa i | is the angle difference of the corresponding internal angles of the triangle, similarity The larger it is, the more similar the triangles are.

[0034] The constraint 1 is:

[0035]

[0036] Formula (5) indicates and are the most similar triangles, and and They are and The second similar triangle;

[0037] V S is 1, ensuring that the matching is not based on random effects, α is the rotation angle of the two triangles, β i (i=1,2,3) is the difference between the angles between the rectangles corresponding to the two semantic feature points and the Y axis, V α is the threshold value;

[0038] Establish corresponding relationships between triangles that satisfy constraint 1 and store them in the set {C1,C2...C X}middle;

[0039] The constraint 2 is:

[0040]

[0041] Among them C a , C b represents two sets of triangle pairs that satisfy constraint 1, η x For C x The scaling factor, η c is η a, η b The mean of |||| is the Euclidean distance between the two triangles, V η 、V E is the threshold value;

[0042] Formula (6) indicates that the triangle set is grouped according to the scaling factor and whether the relative position relationship between the triangles is the same;

[0043] If C a , C b If equation (6) is satisfied, they are grouped together until the set {C1, C2...C X All C in} x Complete the grouping.

[0044] The above step 3 uses the rectangle matching result and the corresponding relationship between the laser point cloud and the image grid to calculate the projection matrix through the least squares principle and the indirect adjustment method, so that any point in the multispectral image becomes a point in the laser point cloud image through the projection rectangle, thereby assigning multispectral information to the laser point cloud, and finally obtaining a laser point cloud with multispectral information, realizing the accurate fusion of the laser point cloud and the multispectral image.

[0045] The above-mentioned laser point cloud and multispectral image fusion method combining Hough transform and semantic feature points is characterized in that the projection matrix is ​​calculated by the least squares principle and indirect adjustment method using the rectangular matching result and the corresponding relationship between the laser point cloud and the image grid, as follows:

[0046] The laser point cloud is used as the image to be matched, and the multispectral image is used as the matching image. i '=(X MS ,Y MS ,1) T After being mapped by the projection matrix A, it becomes f in the laser point cloud image i =(X PC ,Y PC ,1) T , the mapping relationship is:

[0047]

[0048] where h i (i=1,2,...,8) are the parameters in the projection matrix;

[0049] Match several pairs of rectangular box discrete points f i '=(X MS ,Y MS ,1) T 、f i =(X PC ,YPC ,1) T Substituting into formula (7), using the least squares principle and the indirect adjustment method, the eight parameters of the projection matrix A can be solved iteratively.

[0050] The present invention has the following beneficial effects:

[0051] The method of the present invention combines Hough transform with semantic feature point matching, and effectively solves the problem of cross-modal data matching and fusion in complex terrain scenes.

[0052] The present invention effectively considers the problems of different scales, relative rotation, geometric correction, radiation difference, etc. in the cross-data fusion process. The rectangular frame in the line feature image is extracted by Hough transform, and the semantic feature points are constructed by using the center of gravity of the rectangle and various attribute information. The matching between the rectangular frames is realized by using constraint relationships, thereby realizing the matching and fusion of the two-modal data. The robustness and robustness of the matching and fusion are significantly improved, and a new solution is provided for the fusion of multi-source remote sensing data.

[0053] The present invention can be applied to cross-modal data fusion in complex land feature scenes, and can be applied to the fields of accurate identification and classification of urban land features, disaster assessment and rescue, and cultural relics recording and protection, providing richer data support for obtaining fine features of cities and environments, and cultural relics information. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of the method of this embodiment;

[0055] Figure 2 is the image rectangle detection result diagram of the scene in this embodiment;

[0056] Figure 3 is a matching result diagram of the laser point cloud and multispectral image of scenes 1-4 in this embodiment;

[0057] Figure 4 It is the global fused image of the scene in this embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0059] Although the steps in the present invention are arranged with numbers, they are not used to limit the order of the steps. Unless the order of the steps is clearly stated or the execution of a certain step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" used in this article involves and covers any and all possible combinations of one or more of the associated listed items.

[0060] refer to Figure 1 The present invention proposes a laser point cloud and multispectral image fusion method combining Hough transform and semantic feature points, which is suitable for cross-modal fusion of laser point cloud and multispectral image in complex ground scene, including:

[0061] Step 1: Input the laser point cloud image and multispectral image into the Line-CNN deep learning network to extract the line features of the corresponding images, and obtain the point cloud feature straight line graph and the multispectral feature straight line graph;

[0062] Step 2: Map the point cloud feature straight line graph and the multispectral feature straight line graph to the Hough space, and obtain the rectangular centroid and attribute information of the point cloud and the rectangular centroid and attribute information of the multispectral through peak point detection as the semantic feature points of the point cloud and the multispectral semantic feature points;

[0063] Step 3: Rectangular matching is performed on the semantic feature points of the point cloud and the multispectral semantic feature points. The projection matrix is ​​calculated based on the rectangular matching results. The multispectral information is assigned to the laser point cloud based on the projection matrix. Finally, a laser point cloud with multispectral information is obtained, realizing the precise fusion of the laser point cloud and the multispectral image.

[0064] In the embodiment, the technical solution of the present invention can be implemented by computer software to achieve automatic operation.

[0065] The step 1 rasterizes the laser point cloud to convert it into an image, and extracts the characteristic straight lines of the images in the two modes through the Line-CNN deep learning network. The characteristic straight line extraction based on the Line-CNN deep learning network includes:

[0066] Step 101: Converting laser point cloud to image: rasterizing the laser point cloud data and establishing a corresponding relationship to convert it into a two-dimensional raster image;

[0067] The embodiment converts the laser point cloud data into a two-dimensional image by rasterization. First, a grid space is defined, and the grid unit size is set to 0.5m. When there are one or more point clouds in a grid unit, the attribute value of the grid unit is assigned using the linear triangulation method; when there are no point clouds in the grid unit, the attribute value of the grid unit is assigned using the interpolation method, thereby obtaining a relatively complete grid image.

[0068] Step 102: Line feature extraction of bimodal images: Line feature extraction of bimodal images is performed based on the Line-CNN neural network in four stages: feature extraction, node prediction, line segment sampling, and line segment correction. The line feature results are drawn on a blank image of the same size as the original image.

[0069] The embodiment inputs the two-modal images into the four stages of feature extraction, node prediction, line segment sampling, and line segment correction in the Line-CNN neural network, and finally outputs the line feature extraction results of the laser point cloud image and the multispectral image, and then draws the line features on a blank image of the same size as the original image, so that the subsequent rectangular frame detection is not interfered by other image parts.

[0070] Step 2 maps the feature line image to the Hough space, finds the rectangle formed by the feature line according to the peak point distribution law of the rectangle mapping in the Hough space, removes the pseudo rectangle and returns the center of gravity and attribute information of the rectangle as the semantic feature point. The specific extraction of the rectangle frame based on Hough transform is as follows:

[0071] Hough transform detection of rectangular frame: The line feature image extracted from the two-modal image is transferred to the Hough space, all peak points greater than the threshold are counted, several pairs of approximately parallel and equal straight line segments are found, and then all pairs of straight line segments are compared to match the four sides of the rectangle, and pseudo rectangles are eliminated. The rectangle information is returned, and the center of gravity of the rectangle is used as the semantic feature point, and the corresponding rectangle attribute information is assigned to it;

[0072] In order to find the rectangle formed by the straight line segments, it is found that the distribution of the four sides of a rectangle in the image space in the Hough space has the following rules: 1. The first pair of points HS1 and HS2 appear at θ = β1, and the second pair of points HS3 and HS4 appear at θ = β2, and |β1-β2| = 90°, that is, the angle between the adjacent sides of the rectangle is a right angle; 2. When θ is equal, the cumulative value of the same pair of peak points is equal, that is, Peak HS1 (ρ1,θ1)=Peak HS2 (ρ2,θ2)=W,Peak HS3 (ρ3,θ3)=Peak HS4 (ρ4,θ4)=L, that is, the lengths of the opposite sides of the rectangle are equal; 3. When θ is equal, the distance between the same pair of peak points in the ρ axis direction is equal to the length of the side of the rectangle, that is, |ρ1-ρ2|=L, |ρ3-ρ4|=W. Using these characteristics, we can detect rectangles in the image space.

[0073] In order to find several pairs of approximately parallel and equal straight line segments, we perform permutation operations on all the peak points counted. Assume that a pair of peak points is HS i (ρ i ,θ i ), HS j (ρj ,θ j ), the corresponding peak point cumulative value is Peak HSi (ρ i ,θ i ),Peak HSj (ρ j ,θ j ), if the following equation is satisfied, they are matched as a pair of approximately parallel and equal straight line segments:

[0074]

[0075] Where V θ 、V Peak is a very small positive threshold. The two points that satisfy equation (1) are marked as in

[0076] Finally, compare all pairs k (k=1,2,3,...), the pair that satisfies the following conditions k , pair l Match the four sides of a rectangle:

[0077]

[0078] in, The above method only uses the direction and length information of the straight line, so a pseudo rectangle may appear. The coordinates of the four vertices of the rectangle A1, A2, A3, and A4 are obtained based on the two sets of opposite sides of the rectangle. The number of pixels near the four sides is counted and compared with the corresponding side lengths of the rectangle. If any side does not meet the following conditions, it is judged as a pseudo rectangle and removed:

[0079] |P ij -ζ k |<V P ζ k (3)

[0080] Where P ij A is a straight line segment i A j The number of nearby pixels, V P is the ratio threshold.

[0081] In the embodiment, the bimodal line feature extraction image is input into the Hough space, converted into a Hough space image, all peak points greater than the threshold are counted, and stored as a set of n peak points, and the arrangement operation in formula (1) is performed on them two by two to match a number of parallel and equal straight line pairs, and the two pairs of straight lines that satisfy formula (2) between all the straight line pairs are matched as the four sides of the rectangle to obtain a number of groups of rectangles, and the rectangles that do not satisfy formula (3) are eliminated. The obtained rectangle detection result is as follows: Figure 2 As shown. Return the center of the rectangle as the semantic feature point, and assign the corresponding rectangle information to the semantic feature point.

[0082] Rectangular matching is performed based on semantic feature points, and conversion parameters are calculated to achieve the fusion of laser point cloud and multispectral image, and finally a laser point cloud with multispectral information is obtained. Rectangular matching and cross-modal data fusion based on semantic feature points. The details are as follows:

[0083] (1) Semantic feature point matching rectangle: All semantic feature points are exhaustively arranged and combined into triangles, and constraint matching similar triangles are established and classified. After classification, the group with the largest number of triangle pairs is taken as the matching result.

[0084] In order to evaluate the similarity of two triangles, the present invention uses the following formula to calculate the similarity between triangles:

[0085]

[0086] in, and are the triangle sets formed by the semantic feature points of point cloud data and multispectral data, respectively, |Δa i | is the angle difference of the corresponding internal angles of the triangles. The larger S is, the more similar the two triangles are. Thus, constrained matching and classification of similar triangles are established:

[0087] Constraint 1:

[0088]

[0089] express and are the most similar triangles, and and They are and The second similar triangle, V S is 1, ensuring that the matching is not based on random effects. α is the rotation angle of the two triangles, β i (i=1,2,3) is the difference between the angles between the rectangles corresponding to the two semantic feature points and the Y axis, V α is a very small threshold, indicating that α and β i (i=1,2,3) are approximately equal. Establish corresponding relationships between triangles that satisfy constraint 1 and store them in the set {C1,C2...C X}middle.

[0090] Constraint 2:

[0091]

[0092] where η x For Cx The scaling factor, η c is η a , η b The mean of |||| is the Euclidean distance between the two triangles, V η 、V E is a very small threshold. If C a , C b If formula (6) is satisfied, they are divided into the same group until all C x Complete the grouping.

[0093] In the embodiment, all semantic feature points are exhaustively arranged and combined into triangles, and the similarity between triangles is calculated according to formula (4), and the other two triangles that are most similar and second most similar to each triangle are found. The triangles that satisfy formula (5) are established in correspondence and stored in a matching set. Finally, the triangle pairs in the matching set are grouped according to formula (6). The group with more matching pairs has a greater possibility of containing true matching pairs. The group with the largest number of matching pairs is taken as the result of rectangle matching. The matching result is as follows: Figure 3 shown.

[0094] (2) Fusion of laser point cloud and multispectral image: Based on the matching results, the mapping relationship between the laser point cloud and the multispectral image is established. Then, combined with the corresponding relationship between the laser point cloud and the image grid, the projection matrix is ​​calculated, and the multispectral information in the multispectral image is assigned to each laser point cloud. Finally, the accurate fusion of the laser point cloud and the multispectral image is achieved.

[0095] In order to achieve accurate fusion of the two-modal data, the projection rectangle is used to establish the mapping relationship between the laser point cloud and the multispectral image. The laser point cloud is used as the image to be matched, and the multispectral image is used as the matching image. i '=(X MS ,Y MS ,1) T After being mapped by the projection matrix A, it becomes f in the laser point cloud image i =(X PC ,Y PC ,1) T , the mapping relationship is:

[0096]

[0097] where h i (i=1,2,...,8) are the parameters in the projection matrix. Several pairs of matched rectangular discrete points are substituted into formula (7), and the 8 parameters in the projection matrix A are iteratively solved by using the least squares principle and indirect adjustment method to achieve accurate fusion between point cloud and image.

[0098] The embodiment uses the rectangle matching result and the correspondence between the laser point cloud and the image grid to calculate the projection matrix in formula (7) through the least squares principle and the indirect adjustment method, so that any point in the multispectral image becomes a point in the laser point cloud image through the projection rectangle, thereby assigning multispectral information to the laser point cloud, and finally obtaining a laser point cloud with multispectral information, realizing the accurate fusion of the laser point cloud and the multispectral image. The fusion result of the laser point cloud and the multispectral image in the complex terrain scene is as follows: Figure 4 shown.

[0099] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0100] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A laser point cloud and multispectral image fusion method combining Hough transform and semantic feature points, characterized in that: include: Step 1: Input the laser point cloud image and multispectral image into the Line-CNN deep learning network to extract the line features of the corresponding images, and obtain the point cloud feature straight line graph and the multispectral feature straight line graph; Step 2: Map the point cloud feature straight line graph and the multispectral feature straight line graph to the Hough space, and obtain the rectangular centroid and attribute information of the point cloud and the rectangular centroid and attribute information of the multispectral through peak point detection as the semantic feature points of the point cloud and the multispectral semantic feature points; Step 3: Rectangular matching is performed on the semantic feature points of the point cloud and the multispectral semantic feature points. The projection matrix is ​​calculated based on the rectangular matching result. The multispectral information is assigned to the laser point cloud based on the projection matrix. Finally, a laser point cloud with multispectral information is obtained, and the accurate fusion of the laser point cloud and the multispectral image is realized. The step 3 performs rectangular matching on the point cloud semantic feature points and the multispectral semantic feature points, including: All semantic feature points are exhaustively arranged and combined into triangles, the similarity between triangles is calculated, and the other two triangles that are most similar and second most similar to each triangle are found. The triangles that meet constraint 1 are established in correspondence and stored in the matching set. Finally, the triangle pairs in the matching set are grouped according to constraint 2. The group with more matching triangle pairs is more likely to contain true matching pairs. The group with the most matching pairs is taken as the result of rectangle matching. The calculation formula for the similarity between the triangles is: in, and are the triangle sets formed by the semantic feature points of point cloud and multispectral semantic feature points, respectively. i | is the angle difference of the corresponding internal angles of the triangle, similarity The larger it is, the more similar the triangles are. The constraint 1 is: Formula (5) indicates that T i P and are the most similar triangles, and T e M and T i P and The second similar triangle; V S is 1, ensuring that the matching is not based on random effects, α is the rotation angle of the two triangles, β i (i=1,2,3) is the difference between the angles between the rectangles corresponding to the two semantic feature points and the Y axis, V α is the threshold value; Establish corresponding relationships between triangles that satisfy constraint 1 and store them in the set {C1,C2...C X }middle; The constraint 2 is: Among them C a , C b represents two sets of triangle pairs that satisfy constraint 1, η x For C x The scaling factor, η c is η a , η b The mean of |||| is the Euclidean distance between the two triangles, V η 、V E is the threshold value; Formula (6) indicates that the triangle set is grouped according to the scaling factor and whether the relative position relationship between the triangles is the same; If C a , C b If equation (6) is satisfied, they are grouped together until the set {C1, C2...C X All C in} x Complete the grouping; Described step 3 utilizes the corresponding relationship between rectangle matching result and laser point cloud and image grid, calculates projection matrix by least square principle and indirect adjustment method, makes any point in multispectral image become point in laser point cloud image through projection rectangle, thereby multispectral information is assigned to laser point cloud, finally obtains laser point cloud with multispectral information, realizes accurate fusion of laser point cloud and multispectral image; The projection matrix is ​​calculated by using the rectangle matching result and the correspondence between the laser point cloud and the image grid through the least squares principle and the indirect adjustment method, as follows: The laser point cloud is used as the image to be matched, and the multispectral image is used as the matching image. i '=(X MS ,Y MS ,1) T After being mapped by the projection matrix A, it becomes f in the laser point cloud image i =(X PC ,Y PC ,1) T , the mapping relationship is: where h i (i=1,2,...,8) are the parameters in the projection matrix; Match several pairs of rectangular box discrete points f i '=(X MS ,Y MS ,1) T 、f i =(X PC ,Y PC ,1) T Substitute into formula (7), use the least squares principle and indirect adjustment method to iteratively solve the 8 parameters in the projection matrix A.

2. The laser point cloud and multispectral image fusion method combining Hough transform and semantic feature points according to claim 1, characterized in that: The step 1 comprises: Step 101: rasterizing the laser point cloud image to convert it into a two-dimensional raster laser point cloud image; Step 102: Line feature extraction of laser point cloud images and multispectral images is performed based on the four stages of feature extraction, node prediction, line segment sampling, and line segment correction of the Line-CNN neural network. The line feature results are drawn on a blank image of the same size as the original image to obtain a point cloud feature straight line graph and a multispectral feature straight line graph.

3. The laser point cloud and multispectral image fusion method combining Hough transform and semantic feature points according to claim 1, characterized in that: The step 2 maps the point cloud feature straight line graph and the multispectral feature straight line graph to the Hough space respectively, counts all peak points greater than the threshold in the Hough space, finds out several approximately parallel and equal straight line segment pairs, compares all the straight line segment pairs, matches the four sides of the rectangle, removes pseudo rectangles, returns the center of gravity of the rectangle and assigns it various attribute information of the corresponding rectangle as a semantic feature point.

4. The laser point cloud and multispectral image fusion method combining Hough transform and semantic feature points according to claim 3 is characterized in that: The method of counting all peak points in the Hough space that are greater than a threshold value and finding a number of approximately parallel and equal straight line segment pairs includes: Assume that a pair of peak points are HS i (ρ i ,θ i ), HS j (ρ j ,θ j ), the corresponding peak point cumulative value is Peak HSi (ρ i ,θ i ),Peak HSj (ρ j ,θ j ), if the following equation is satisfied, it is matched as a pair of approximately parallel and equal straight line segments: Where V θ 、V Peak is the positive threshold; The two peak points satisfying equation (1) are marked as The parameters 5. The laser point cloud and multispectral image fusion method combining Hough transform and semantic feature points according to claim 4, characterized in that: The comparison of all pairs of straight line segments to match the four sides of the rectangle includes: Compare all The following conditions will be met Match the four sides of the rectangle: in, is the angle threshold.

6. The laser point cloud and multispectral image fusion method combining Hough transform and semantic feature points according to claim 5, characterized in that: The pseudo rectangle removal is specifically as follows: The coordinates of the four vertices of the rectangle are obtained based on the two sets of opposite sides of the rectangle, A1, A2, A3, and A4. The number of pixels near the four sides is counted and compared with the corresponding side lengths of the rectangle. If any side does not meet the following conditions, it is judged as a pseudo-rectangle and removed: |P ij -g k | <V P g k (3) Where P ij For side A i A j The number of nearby pixels, V P is the ratio threshold.

Images